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PREFACE 


This  panel  was  a  continuation  of  the  documentation  panel  of 
the  first  software  workshop  held  In  April  1979.  The  original 
panel  examined  the  existing  software  documentation  arena,  and 
developed  overview  descriptions  for  whot  they  considered  to  be  e 
comprehensive  set  of  documents  that  rould  be  Msed  fer  EhD  contracts. 
Their  recommendations  for  further  action  Included' 

1.  Preparation  of  Data  Item  Descriptions  (DIDs)  for  these 
documents . 

2.  Preparation  of  modifications  to  existing  military 
standards  (MIl-STDs),  or  the  creation  of  a  new  one, 
from  which  the  DIDs  can  be  Involved. 

3.  Preparation  of  guidelines  to  help  program  managers 
select  an  appropriate  subset  of  documents  for  specific 
contracts. 

Since  the  1979  workshop,  the  JLC  completed  the  first  task  vie 
contract,  and  awarded  a  contract  to  accomplish  the  second  task. 


SOFTWARE  DOCUMENTATION 


1 .  OBJECTIVE 

During  the  JLC  Software  Workshop  (Monterey  II)  neld  In  Monterey, 

CA  on  22-25  June  1981,  Panel  A,  Software  Documentation,  pursued  four 
objectives: 

a)  Provide  recommendations  to  the  JLC  for  developing  project 
management  guidelines  for  the  selection  of  software 
documentation. 

b)  Clarify  the  relationship  between  the  DOD  acquisition  life-cycle 
(milestones,  phases)  and  the  JLC-llst  of  software  documents. 

c)  Provide  recommendations  to  the  JLC  concerning  the  addition, 
deletion  or  modification  of  documents  In  the  JLC-llst  of 
software  documents. 

d)  Provide  recommendations  to  the  JLC  for  Implementing  the  standard 
set  of  software  documents  (JLC-llst)  within  the  OSD/JLC/Services. 

The  "JLC"  referenced  In  the  above  objectives  refers  more  directly  to  the 
Computer  Software  Management  (CSM)  Subgroup  of  the  Joint  Logistics 
Commanders  Joint  Policy  Coordinating  Group  on  Computer  Resource  Manage¬ 
ment  (JLC-JPCG-CRM) . 

2.  SCOPE 

The  objectives  of  the  Software  Documentation  Panel  initially  provided 
by  the  CSM  Subgroup  were: 

a)  Develop  guidelines  for  project  managers  to  help  select  the 
appropriate  subset. 

b)  Prepare  draft  implementation  plan  for  the  DIDs  and  recommend 
method  of  evaluation. 

The  pursued  objectives  differed  from  these  Initial  objectives,  for  two 
reasons.  First,  the  time  (three  days)  available  at  Monterey  II  was 
insufficient  to  meet  the  objectives.  Secondly,  it  became  apparent,  at 
the  beginning  of  the  Panel  session,  that  several  issues  needed  to  be 
clarified  or  resolved  in  order  to  meet  the  more  long-term  objectives 
of  acquisition  management  guidelines  and  DID  Implementation  plans. 
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It  was  felt  that  acquisition  management  guidelines  are  certainly 
necessary,  but  that  Panel  A's  contribution  to  the  CSM  Subgroup  should 
be  to  clarify  the  issues  involved  with  selecting  documents.  The  major 
supporting  panel  tasks  decided  upon  were  developing  a  consistent 
acquisition  cycle  model  and,  concurrently,  reviewing  the  DID's  produced 
as  a  result  of  the  CSM  Subgroup's  documentation  contract.  The  information 
resulting  from  panel  deliberations  on  these  tasks  was  felt  to  be  an 
integral  part  of  any  guidelines.  It  was  also  felt  that  contractual  support 
is  necessary  for  guideline  development  and  to  this  end  a  discussion  of 
the  required  tasking  was  undertaken. 

Preparing  a  draft  implementation  plan,  per  se,  was  not  tackled. 

It  was  felt  that  the  approach  most  beneficial  to  tne  CSM  Subgroup  would 
be  to  document  some  of  the  considerations  involved  in  implementation  of 
the  DID's. 

Two  additional  topics  were  considered  for  discussion  but  were  dropped 
because  they  were  not  felt  to  be  within  the  scope  of  documentation/DID 
specification,  standardization  and  tri-service  implementation.  These 
topics  were  the  automation  of  document  generation  and  the  proper  specification 
of  software  requirements.  These  are  important,  related  technological  issues 
but  may  not  have  a  place  in  JLC  standardization. 

Twelve  members  of  the  panel  met  at  Ft.  Belvlor,  VA  in  August  to  review 
the  draft  Final  Report  and  complete  the  DID  review.  This  report  incorporates 
the  results  of  the  follow-up  meeting. 

3.  APPROACH 

Discussions  at  the  Initial  panel  meeting  revealed  several  deep-seated 
concerns  about  the  documents,  the  DIDs  and  the  life  cycle.  These  concerns 
led  to  the  formation  of  four  subpanels  which  addressed  specific  issues. 

The  subpanels  met  independently  throughout  the  remainder  of  the  workshop. 
Periodic  full  panel  sessions  were  held  to  coordinate  the  activity  and 
develop  group  concensus. 

The  four  subpanels  were: 

1.  Guidelines  for  Acquisition  Managers. 

2.  Life  Cycle. 
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3.  Document  set  and  DIDs. 

4.  Implementation  Plan. 

Membership  of  the  subpanels  is  given  in  Appendix  A. 

Two  of  the  subpanels  had  extensive  interaction  with  representatives 
of  the  CSM.  The  Life  Cycle  was  a  particularly  controversial  topic  and, 
during  one  session,  discussion  between  subpanel  and  CSM  members  was  needed 
to  clarify  the  intent  of  the  new  life  cycle. 

The  implementation  subpanel  attended  a  CRM-CSM  meeting  on  policy 
because  the  enforcement  of  the  new  DIDs  requires  promulgation  of  the  new 
MIL-STD.  This  helped  formulate  the  conclusions  reached  by  this  subpanel. 

4.  DISCUSSION 

4.1  SUBPANEL  1:  GUIDELINES  FOR  ACQUISITION  MANAGERS 

This  subpanel  concluded  at  the  onset  that  there  was  not  sufficient 
time  during  the  workshop  to  develop  a  set  of  guidelines.  Instead,  they 
decided  to  recommend  that  the  guidelines  themselves  be  developed  via 
contract,  and  directed  their  efforts  at  drafting  a  Statement  of  Work  (SOW) 
for  such  an  effort.  (See  Appendix  C). 

As  part  of  their  deliberations,  subpanel  1  considered  two  major 
questions: 

1.  What  flexibility  should  an  acquisition  manager  have  in  tailoring 
Individual  DIDs? 

2.  What  determines  if  an  individual  DID  is  needed  on  a  particular 
project? 

Regarding  the  question  of  tailoring,  the  subpanel  agreed  (after  considerable 
discussion)  that  DID  tailoring  should  be  discouraged.  The  basis  for  this 
decision,  which  differed  from  that  of  the  Monterey  I  panel,  was  that  document 
standardization  Is  the  most  Important  concept  behind  the  use  of  a  single 
DID  set,  and  that  prohibition  of  tailoring  does  not  prohibit  offerors  or 
acquisition  managers  from  stating  particular  sections  are  not  applicable 
if  such  is  the  case. 


4 


Regarding  the  question  of  DID  selection,  the  subpanel  had  three 
concerns.  First,  it  was  agreed  that  there  was  considerable  overlap  in 
information  content  of  some  of  the  documents.  Second,  it  was  felt 
that  not  all  of  the  documents  were  necessary  on  every  project.  Third, 
it  was  feared  that  acquisition  managers,  when  in  doubt  as  to  what 
documents  to  require,  would  tend  to  procure  all  of  them. 

In  view  of  these  concerns,  the  subpanel  first  decided  that  some  sort 
of  selection  matrix  which  would  list  appropriate  DIOs  versus  project 
characteristics  was  needed.  However,  in  a  later  split  decision,  the 
panel  concluded  that  a  flow  chart  or  selection  algorithm  would  be  better 
than  a  decision  matrix.  (Minority  opinions  are  contained  in  Section 
4.1.2).  Despite  the  differences  in  opinion  regarding  the  form  of  the 
selection  technique,  the  subpanel  unanimously  agreed  that  the  tech¬ 
nique  should  be  incorporated  into  a  guidebook. 

4.1.1  Guidebook 

The  subpanel  believed  such  a  guidebook  should  contain  the  following: 

1.  A  selection  matrix,  flow  diagram,  or  other  algorithmic  form. 

2.  General  discussion  of  why  documentation  is  necessary,  with 
case  histories  and  the  effect  of  documentation  on  their 
success  or  failure. 

3.  Statements  indicating  how  military  standards  and  regulations 
relate  to  documentation  and  which  are  in  effect. 

4.  Descriptions  of  the  role  of  the  Statement  of  Work  (SOW)  and 
Contract  Data  Requirements  List  (CDRL)  in  requiring  DIDs  and 
providing  flexibility  when  appropriate. 

5.  Expanded  overviews  of  each  document. 

6.  Project  characteristics  to  be  considered  in  selection  DIDs  to 
be  procured. 

Some  of  the  project  characteristics  which  the  subpanel  felt  should 
be  included  in  the  guidebook  were 

1.  Number  of  project  offices,  services,  and  contractors  involved 

2.  Complexity  factors,  such  as  multiple  processors  and  sizing/timing 
constraints 

3.  Criticality  of  software  to  system  operation 
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4.  Interoperability  requirements 

5.  Stability  of  requirements  over  system's  life 

6.  Projected  length  of  system  life  cycle 

7.  Budget  constraints 

8.  Use  of  existing  software  and  documentation 

9.  Procurement  type  (sole  source  vs.  multiple) 

10.  Software  support  concepts  (contractor  vs.  organic) 

11.  Development  phase 

12.  Significance  of  software  In  overall  system,  i.e.. 

Hardware  Intensive,  Software  Intensive,  Operator  intensive 

4.1.2  Minority  Reports 

Two  Minority  Reports  were  submitted. 

4. 1.2.1  Minority  Report  1 

1.  The  subpanel  is  recomnending  that  a  task  be  let  to  a  contractor 
to  put  together  a  Guidebook  to  aid  Program  Managers  in  the 
selection  of  OIDs/Documer.ts  In  support  of  Software/Firmware 
development. 

2.  As  part  of  the  guidebook  an  Algorithm  may  be  required  to  be 
developed  that,  based  on  characteristics  of  the  project,  will 
lead  a  Program  Manager  to  his  selection  of  DIDs. 

The  subpanel  feels  that  this  type  of  Algorithm  is  needed  because 
of  the  overlapping  coverage  of  some  DIDs 

3.  The  minority  analysis  of  the  problem  leads  to  the  conclusion  that: 

a)  The  number  of  DIDs  has  been  reduced  to  a  workable 
set. 

b)  A  minor  expansion  of  the  DID  Overview  Descriptions 
would  suffice  in  the  place  of  an  Algorithm  and  be 
simpler  to  use  than  a  flow  type  algorithm.  If  all 
additional  information  were  provided  in  the  DID 
overviews  such  as: 

1.  Under  what  conditions  is  the  DID 
needed  and  why? 

2.  Under  what  conditions  is  the  DID  not 
needed? 

3.  Is  there  a  paragraph/paragraphs  from 
another  DID  that  relate  to  this  DID 
that  could  satisfy  this  need? 
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c)  A  contract  may  not  have  to  be  let  to  accomplish  this  - 
It  may  be  within  the  scope  of  the  OLC  staff  to 
accomplish  in-house. 

d)  It  is  recommended  that  a  complete  set  of  software/ 
firmware  OIDs  be  attached  to  the  Guidebook  along 
with  the  "expanded  overviews"  to  give  the  Program 
Manager  all  he  needs  to  make  the  document  selection. 

e)  It  is  felt  the  algorithm  is  not  impossible  to  construct, 
but  may  be  infeasible  due  to  the  estimated  6  months 

to  develop  it  and  then  validate  it  and  due  to  the 
difficulty  in  quantifying  project  characteristics  in 
a  way  that  can  be  applied  to  an  algorithm  and  due  to 
the  number  of  variables  involved. 

4. 1.2. 2  Minority  Report  2 

A  second  minority  report  also  disagreed  with  the  development  of  an 
"algorithm"  for  DID  selection.  The  implication  of  an  "algorithm"  is 
that  DID  selection  can  be  reduced  to  a  series  of  black  and  white 
decisions,  the  results  of  which  totally  determine  what  OIDs  are  "appro¬ 
priate."  However,  as  alluded  In  1  to  3  above,  the  "number  of  variables" 
Involved  may  be  too  great  to  render  a  neat,  deterministic  selection 
process  feasible.  The  selection  process  thus  becomes  a  complex  one  of 
synthesis,  judgement,  and  in  part.  Intuition. 

Thus,  the  inclusion  of  an  "algorithm"  poses  a  real  risk  of  leading 
the  uninitiated  acquisition  manager  to  conclude  the  DID  selection  process 
is  less  complex  than  It  really  is.  A  "good"  acquisition  manager  who 
cannot  decide  for  himself/herself  what  DIDs  are  needed  should  delegate 
the  authority  to  someone  on  the  staff  who  has  the  background  to  make 
the  decision.  If  the  acquisition  manager  does  not  have  access  to  some¬ 
one  who  does  have  the  background,  he/she  should  hire  someone  who  does, 
or  get  out  of  the  software  acquisition  business,  if  the  latter  is  not 
possible,  then  require  all  the  DIDs.  (From  the  perspective  of  risk  and 
life  cycle  implications,  the  public  interest  is  perhaps  better  served 
by  ordering  and  paying  for  a  few  DIDs  which  are  not  really  needed  in 
the  final  analysis  than  by  not  ordering  DIDs  which  eventually  are 
needed.)  In  any  event,  let  us  not  try  to  reduce  what  is  often  truly 
a  complex  and  somewhat  judgemental  decision  to  an  algorithm. 

4.2  SUBPANEL  2:  LIFE  CYCLE 

The  DID  set  developed  for  the  JLC  is  based  on  a  new  software  life 
cycle  which  includes  two  new  reviews:  the  Software  Specification  Review 
(SSR)  and  the  Test  Readiness  Review  (TRR).  Initial  deliberations  by  this 


subpanel  centered  on  the  necessity  of  these  new  reviews.  After  much 
heated  discussion,  the  subpanel  decided,  in  a  split  decision,  that 
the  question  of  whether  or  not  these  new  reviews  were  necessary  was 
not  their  charter.  They  then  proceeded  to  examine  the  role  of  the 
documents  in  the  life  cycle,  with  the  ground  rule  that  the  life  cycle 
(including  SSR  and  TRR)  is  "given." 

The  subpanel  examined  all  documents  in  the  JLC  package  and  pro¬ 
jected  them  against  the  life  cycle  chart,  showing  when  each  should  be 
available  in  preliminary  form,  baselined,  or  modified.  (This  projection 
was  accomplished  under  the  handicap  of  not  having  a  formal  description 
of  the  life  cycle  available  to  the  panel).  The  life  cycle  is  shown 
in  figure  1 . 

Additional  conclusions  drawn  by  the  subpanel  Include: 


1.  The  life  cycle  chart  should  reflect  the  parallel  paths  for 
hardware  and  system  development.  Software  should  always 
be  considered  a  part  of  the  overall  system.  The  phases  of 
Hardware  and  Software  must  permit  consideration  of  decision 
processes. 

2.  The  life  cycle  should  be  tailored  to  specific  projects  and 
phases  of  development.  The  life  cycle  figure  indicates 
the  software  acquisition  process  which  is  key  to  a  total 
system  acquisition.  This  process  may  occur  within  any 
phase  of  the  system  development  life  cycle.  However,  the 
reviews,  documents  and  phases  may  be  tailored  as  required 
by  the  complexity  of  the  development. 

3.  The  life  cycle  model  can  be  applied  to  any  development 
process,  whether  "start  from  scratch",  modification  to  an 
existing  system  or  the  maintenance  phase. 

4.  A  method  for  proving  a  life  cycle  phase  has  been  completed 
needs  to  be  devised. 

5.  A  good  definition  of  "design  review"  needs  to  be  developed. 
It  was  felt  that  the  reviews  indicated  on  the  life  cycle 
chart  represent  culminations  of  the  review  processes  which 
track  on-going  activities. 

6.  The  new  policy  will  reflect  the  standardization  of  existing 
practices.  A  means  for  providing  evolution  to  incorporate 
new  methodologies  must  be  developed. 
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Figure  1,  Acquisition  Cycle 
ABBREVIATIONS 
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CSDM 

CSOM 

CSUM 

DBDD 

FCA 

FQT 

IDO 


IRS 

PCA 

PDR 


Critical  Design  Review 

Computer  Resource  Life  Cycle  Management  Plan  (Acquisition) 

Computer  Resource  Life  Cycle  Management  Plan  (Support) 

Computer  System  Diagnostic  Manual 

Computer  System  Operator's  Manual 

Computer  System  User's  Manual 

Data  Base  Design  Document 

Functional  Configuration  Audit 

Formal  Qualification  Test 

Interface  Design  Document 

Interface  Requirements  Spec 

Physical  Configuration  Audit 

Preliminary  Design  Review 


SCMP 

SDDD 

SDP 

SDR 

SPCR 

SPM 

SPS 

SQAP 

SRR 

SRS 


Software  Configuration  Management  Plan 
Software  Detail  Design  Document 
Software  Development  Plan 
System  Design  Review 
Software  Problem/Change  Report 
Software  Programmer's  Manual 
Software  Product  Specification 
Software  Quality  Assurance  Plan 
System  Requirements  Review 
Software  Requirements  Specification 


SSPM 

SSR 

SSS 

STLDD 

STP 

STP 

STR 

SUM 

TRR 

VDD 


Software  Standards  and  Procedures  Manual 

Software  Specification  Review 

System/ Segment  Specification 

Software  Top  Level  Design  Document 

Software  Test  Plan 

Software  Test  Procedure 

Software  Test  Report 

Software  User's  Manual 

Test  Readiness  Review 

Version  Description  Document 


4.3  SUBPANEL  3:  DOCUMENT  SET  AND  DIDs 


This  subpanel  was  tasked  with  reviewing  the  results  of  JLC  DID 
development  contract.  The  scope  of  this  review  included  not  only  the 
JLC  Software  Documentation  Report,  but  also  existing  DIDs  and  the 
Monterey  I  Final  Report.  This  was  supplemented  by  panel  members' 
knowledge  of  existing  software  documents.  Panel  members  received 
the  DID  package  prior  to  assembling  in  Monterey,  and  each  participant 
was  asked  to  review  in  depth  one  or  more  specific  DIDs  and  present 
his/her  findings  during  the  workshop. 

The  first  task  of  the  workshop  was  to  review  the  JLC  software 
documentation  list  for  sufficiency.  Open  discussions  were  held  on 
the  necessity  of  adding,  combining  or  deleting  a  document.  The 
following  general  criteria  were  used  to  guide  the  discussion: 

•  Will  this  document  alleviate  or  add  to  a  problem? 

•  Is  this,  topic  adequately  covered  in  another  document? 

After  discussion  on  additions  or  deletions  a  vote  was  taken  on  the 
recommendation  on  a  document  If  it  was  necessary.  An  open  discussion 
on  the  DIDs  was  then  held  but  was  limited  to  general  rather  than 
specific  comments  for  each  DID  due  to  time  limitations.  Specific 
comnents  were  provided  later  to  designated  panel  members  who  collated 
the  comments.  The  follow-up  meeting  refined  the  comments  which  are 
included  as  Appendix  F. 

4.3.1  Document  Set 

The  panel  felt  that  three  documents  should  be  deleted  from  the  list, 
some  changes  were  necessary  to  those  remaining,  and  three  should  be 
added.  Descriptions  of  these  are  givan  in  Appendix  D.  The  documents 
to  be  deleted  are: 

9 

R-DID-1 02  Computer  Resource  Life  Cycle  Management  Plan, 

Vol.  I,  Acquisition 

R-DID-1 03  Computer  Resource  Life  Cycle  Management  Plan, 

Vol.  II,  Support 

R-DID-1 07  Software  Problem/Change  Report 
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The  first  two  do  not  belong  in  the  contractor  set  of  documents  or 
cited  in  the  new  MIL-STD  because  they  are  government-generated 
documents.  If  they  are  cited  anywhere  they  might  be  appended  to 
0001  5000.7.9  or  left  to  the  services  procurement  practices. 

The  Software  Problem/Chanye  Report  dees  not  require  a  standard 
form  and,  in  any  case,  t.he  one  suggested  does  not  suffice. 

A  minority  report  on  deleting  certain  additional  documents  is 
given  in  section  4.3.4. 

4-3.1.'  Flrnr/are  Support  Data  Document 

The  panel  was  in  almost  total  agreement  that  the  design  and 
development  of  software  and  firmware  should  be  documented  identically. 
The  consensus  was  that  size  and  complexity  are  the  important  factor 
in  selecting  documentation  for  firmware  just  as  it  is  for  software. 

It  was  recognized  that  there  is  a  basic  difference  in  the  storage 
and  modification  of  firmware  and  that  the  existing  documents  do  not 
clearly  address  these  differences  from  a  firmwars  perspective.  The 
Subpanel  reviewed  the  Firmware  Support  Data  010  (UDI-E-3937-ASD)  and 
decided  that  a  Firmware  Supoort  Data  document  snould  be  added  to  the 
JLC  list  with  this  DID  being  used  as  a  guide. 

A  related  discussion  was  held  on  the  need  for  a  firmware  document 
that  addressed  small  developments  such  as  the  Non-Complex  Computer 
Programs  Specification  found  in  Acpendlx  XVI  of  MIL-STD-483.  After 
discussion  witn  the  Guidebook  subpanel  this  panel  agreed  that  a 
subset  of  the  JLC  list  would  be  sufficient  to  address  the  firmware 
requirements  of  a  small  project  without  creating  a  new  document. 

4 . 3 . 1 . 2  Operational  Concept  Document 

One  of  the  documents  proposed  during  Monterey  I  which  was 
subsequently  dropped  from  the  JLC  list  was  the  Operational  Concept 
Document.  This  document  was  presented  to  the  panel  as  being  a 
potentially  valid  document  in  that  it  fed  back  to  the  government  in 
plain  English  what  the  contractor  understood  about  the  proposed 
system,  including  man-machine  interfaces.  The  proponents  were  willing 
to  admit  that,  in  theory,  section  3.1.7  of  System/ Segment  Specification 
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"Operational  and  Organizational  Concept"  addressed  this  area,  but  because  it, 
was  part  of  a  large  specification  It  had  never  been  adequate'iy  addressed 
In  reality.  The  point  was  made  that  in  a  large  system  there  was  a 
potential  need  for  a  separate  document  covering  this  area.  A  vote  was 
taken,  and  approximately  75X  of  panel  thought  that  the  addition  of  this 
document  to  JLC  list  would  enhance  the  software  documentation  of  some 
large  projects;  therefore  it  would  be  recorrmcnded  that  It  be  added 
to  the  JLC  list. 

4 . 3 . 1 . 3  Requirements  Traceablii ty  Matrix 

This  document  was  proposed  by  the  Air  Force  members.  Its  purpose 
Is  to  tie  together  in  one  place  the  allocation  of  requirements  to  lower 
level  documents.  It  was  conceded  that  each  of  the  B-level  documents 
has  a  requirements  traceability  matrix  back  to  the  system  specification 
and  similarly  C  back  to  B.  In  large  projects  however,  where  there  are 
several  CIs  and  CPCIs,  it  is  a  tedious  and  time  consuming  process  to  dig 
these  out  of  each  document  and  collate  all  these  individual  matricies. 

In  the  discussion  which  followed,  it  was  conceded  that  this  document 
would  be  helpful  to  government  personnel  on  large  projects;  thus,  the 
panel  decided  to  reconmend  that  this  document  be  added  to  the  JLC  list. 

4.3.2  Data  Item  Descriptions 

During  the  discussion  on  the  adequacy  of  the  DIDs,  points  were 
raised  that  the  DIDs  included  policy  requirements  that  should  properly 
be  contained  in  a  MTL-STD.  Members  of  the  panel  who  had  attended 
Monterey  I  discussed  the  difference  between  the  actual  implementation 
of  their  recommendations  and  how  they  had  envisioned  this  implementation. 

It  was  the  intent  of  Monterey  l  that  the  DIDs  would  be  contractual 
documents  that  pointed  to  a  MIL-STD  for  embedded  computer  systems  that 
contained  the  required  format  and  content  for  software  documentation. 

As  things  turned  out  the  MIL-STD  development  lagged  considerably  behind 
the  development  of  the  DIDs.  As  a  result,  the  contractor  who  developed  the 
DIDs  put  requirements  into  them  since  there  wasn’t  any  MIL-STD  to 
reference.  It  was  the  general  consensus  of  the  panel  that  revision 
of  these  DIDs  should  be  integrated  with  the  development  of  the  new 
MIL-STD  for  Embedded  Computer  Systems. 


Specific  comments  on  each  of  the  DIDs  are  included  In  Appendix  F. 

General  comnents  are  provided  below: 

•  In  revising  DIDs  the  improvements  made  to  MIL-STD  483  by 
NOTICE  2  should  be  considered. 

•  All  relevant  existing  DIDs  should  be  reviewed  to  determine 
if  they  can  be  superseded  by  the  proposed  DIDs. 

a  The  word  "paragraph"  should  be  replaced  by  "section,"  and 
contractual  language  should  be  used  (l.e.,  "shall"  rather 
than  "should"). 

e  Include  in  review  the  DIDs  not  originally  considered  by  the 
DID  contractor  (Appendix  E). 

•  DIDs  should  not  be  service  related  or  reference  a  service-unique 
regulation  or  guidebook. 

e  Technical  Performance  Measurement  Report  (DI-S-361 9/5-153)  should 
be  replaced  by  Software  Development  Progress  Report  (UOI-A-21 435) . 

•  The  DIDs  should  be  reviewed  to  ensure  that  they  are  not  too  rigid, 
precluding  flexibility  in  documentation. 

a  Review  the  EIA  DIDs  and  see  if  they  can  be  folded  into  the 
JLC's  DIDs. 

•  Man-machine  interfaces  are  not  adequately  covered. 

e  Interface  Specification  should  be  in  the  specification  tree. 

•  SOFTWARE  DIDs  should  be  retitled  SOFTWARF./ FIRMWARE. 

•  In  Block  9  of  every  DID  add  reference  to  MIL-STD- XXX. 

4.3.3  Future 

The  new  DID  package,  though  solving  many  of  our  present  problems, 
does  not  attempt  to  predict  methodologies  to  soivc  our  future  problems 
such  as  the  ramifications  of  ADA  on  software  development  and  documentation 
Thus,  the  subpanel  felt  that  the  JLC  should  periodically  review  and  update 
MIL-STD-XXX  (DID  package)  to  take  advantage  of  new  trends  in  software 
development/'documentati  on . 

4.3.4  Minority  Report 

a.  R-DID-105  "Software  Configuration  Management  Plan 

This  plan  should  be  incorporated  into  the  overall  configuration 
management  plan  to  insure  that  configuration  management  is  consistent. 
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tnat  the  Impact  of  changes  on  all  parts  of  the  system  are  covered. 

This  can  only  occur  If  the  handling  of  all  ECPs  are  coordinated.  The 
Information  In  this  DID  should  be  added  to  that  for  the  overall  CMP. 

Software  CM  personnel  should  help  develop  a  new  CMP  DID. 

b.  R-DID-111  Software  Top  Level  Design  Document 
R-DID-112  Software  Detailed  Design  Document 
k-DID-115  Software  Product  Specification 

The  Software  Top  Level  Design  Document  and  Software  Detailed 
Design  Document  should  be  combined  to  form  the  Software  product 
spec,  eliminating  2  DIDs.  This  document  could  be  delivered  in  snapshot 
forms,  as  required,  until  the  entire  document  Is  accepted.  The 
Configuration  Management  and  delivery  problems  with  chi s  scheme  are 
what  should  be  addressed. 

4.4  SUBPANEL  4:  IMPLEMENTATION  PLAN 

The  objective  of  this  subpanel  was  to  examine  the  problem  of 
bringing  the  revised  DIDs  developed  under  the  DID  study  panel  to  the 
point  of  official  approval. 

"Official  approval"  was  agreed  to  denote  a  combined  joint  service 
endorsement  of  the  individual  DIDs,  followed  by  approval  at  the  DOD 
level.  With  this  approval  accomplished,  the  DIDs  would  be  entered 
into  the  AMSDL  program  and  maintained  within  the  procedures  presently 
in  place  under  DOD  5000. 1-L. 

The  proper  role  of  the  JLC  in  this  effort  was  examined,  and  it 
was  concluded  that  the  on-going  OSD  Standardization  Program  is  the 
orogram  within  DOC  which  acts  as  the  focal  point  for  administering 
the  approval  of  new  documentation  that  is  destined  for  the  AMSDL 
program. 

The  coordination,  then,  must  occur  between  the  JLC  and  the 
Defense  Material  Specifications  and  Standards  Office  (DMSSO)  which  presently 
acts  as  the  DOD  agent  for  the  standardization  of  documentation  relating 
to  Embedded  Computer  Resources.  Several  copies  of  the  draft  OSD 
Standardization  Program  Plan  were  made  available. 


15 


The  question  arose  as  to  how  the  JLC,  as  presently  organized, 
can  effectively  Interface  with  the  OSD  Standardization  Program.  In 
order  to  Investigate  this  question,  the  suboanel  was  invited  to  join 
a  combined  meeting  of  the  Joint  Policy  Coordinating  Group  and  its 
Computer  Software  Management  Subgroup. 

At  this  meeting,  an  explanation  of  the  DOD  Standardization  Program 
was  offered,  and  several  copies  of  its  draft  Program  Plan  was  made 
available  to  the  JPC6  and  the  CSM  Subgroup. 

The  responsibility  for  resolving  the  question  of  "how  should  the 
JLC  organize  Itself  to  Interface  with  the  OSD  Standardization  Program 
as  well  as  to  administer  their  vertical  service  approval  and  their 
joint  service  Integration"  was  accepted  by  the  CSM  Subgroup  chairman. 

This  task  was  scan  to  be  relevant  for  two  reasons: 

a.  The  OSD  Standardization  Program  Plan  needs  to  be  studied, 
particularly  with  regard  to  its  data  requirements. 

b.  There  Is  no  existing  joint  service  documentation  integration 
agency  of  the  type  envisioned  for  the  JLC. 

Considering  these  factors,  It  was  thought  advisable  for  the  CSMS 
to  examine  possible  document  implementation  plans  at  greater  length 
before  committing  to  firm  milestones.  The  CSM  subgroup  chairman  accepted 
the  responsibility  for  resolving  how  the  JLC  should  organize  to  inter¬ 
face  with  the  Standardization  Program  as  well  as  to  administer  inter  and 
intra  service  approval. 

It  was  agreed  that  DIDs  and  their  corresponding  source  MIL  SPECs/ 
Standards/Handbooks  must  be  treated  in  an  integrated  fashion  by  the  JLC 
in  securing  DOD  approval. 

It  was  also  agreed  that  the  ultimate  plan  for  document  implementation 
should  validate  the  effectiveness  by  choosing  on-contract  pilot  projects. 
Another  desirable  feature  of  the  plan  would  be  to  consider  phasing  DID/ 
Standard  sets  by  groups  as  they  become  available.  In  this  way  the  number 
of  new  documentation  requirements  that  are  simultaneously  mandated  can 
be  controlled.  Sample  documents  should  be  developed  as  part  of  a  guidebook. 
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5.  RECOMMENDATIONS 


1.  The  JLC-JPCGCRM-CSM  should  publish  a  document  selection  guidebook 
for  use  by  acquisition  managers  based  on  the  outline  In  Appendix 
C.  .This  guideline  should  address  both  software  (operational, 
support,  ...)  and  firmware. 

2.  Another  guidebook  should  be  developed  to  show  how  to  apply 
the  forthcoming  MIL-STO  to  specific  acquisitions.  Specific 
Items  to  be  addressed  should  Include  the  goal,  conduct,  and 
tailoring  of  reviews.  The  guidebook  should  clarify  what  is 
meant  by  successful  attainment  of  each  life  cycle  phase,  and 
when  documents  are  accepted.  Emphasis  in  the  life  cycle  should 
be  placed  on  what  is  to  be  accomplished  rather  than  who  does 

it  (as  in  current  life  cycle  models).  Three  specific  issues 
should  be  addressed: 

a.  Insurance  that  hardware  and  software  development 
are  coordinated.  This  coordination  should  be 
addressed  at  all  reviews,  especially  at  non-system 
level  reviews. 

b.  Insurance  that  software  and  other  system  efforts 
do  not  duplicate  each  other. 

c.  Firmware  should  be  treated  as  software  during 
development. 

3.  Add  three  new  documents  to  OtC  list:  Requirements  Traceability 
Matrix,  an  Operational  Concept  Document  and  a  Fimwara  Support 
Data  Document. 

4.  Delete  three  documents:  CRICMP  Vol .  I,  CRLCMP  Vol.  II,  Software 
Problem/Change  Report. 

5.  Retitle  SOFTWARE  DIDs  as  "SOFTWARE/ FIRMWARE" . 

t>.  integrate  revision  of  DIDs  with  the  on-going  development  of  a 
MIL-STU  for  Embedded  Computer  Systems.  The  new  DIDs  should  be 
published  in  conjunction  with  the  proposed  DOD  standards  and 
industry  and  the  services  should  have  at  least  one  year  to 
respond  with  comments.  Until  key  issues  are  resolved,  existing 
DIDs  should  not  arbitrarily  be  cancelled. 
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7.  Address  panel's  conments  (Appendix  F)  during  revision  of  DIDs. 
Further,  this  panel  should  remain  active  until  a  published  set 
of  the  new  DIDs  has  been  field  tested  and  key  issues  have  been 
resolved. 

8.  Initiate  an  effort  to  investigate  the  ramifications  of  ADA  and 
other  future  innovations  on  software  development  and  documentation. 

9.  Designate  the  CSM  subgroup  as  the  JLC  agent  responsible  for 
integrating  JLC  standardization  efforts  with  the  OSD  Standard¬ 
ization  Program.  Flow  should  be: 

CSM  — - ►  DMSSO  OSD  "  ►  Service  Secretary 

10.  Transition  to  the  new  MIL-STD  and  DIDs  should  be  done 
using  a  pilot  program  In  conjunction  with  a  phased 
approach. 

11.  The  life  cycle  should  be  reexamined  and  that  current 
trends,  lessons  learned  and  practical  aspects  of  software 
development  be  incorporated.  This  should  include  general¬ 
ization  of  the  life  cycle,  combining  of  phases  and/or 
deletion  of  reviews.  Someone  (OSD?,  JLC?,...?)  must  define 
what  constitutes  successful  attainment  of  the  objectives 

of  life  cycle  phases,  reviews  and  document  deliveries. 


12.  A  parallel  activity  should  be  started  to  evolve  a  successor  § 

policy  and  its  implementation  to  reflect  the  evolution  j 

of  existing  practices  and  technologies  such  as  utilization  I 

of  automated  documentation  techniques.  1 

13.  Guidance  should  be  provided  for  the  preparation  of  documents  | 

according  to  the  DIDs,  including  examples.  I 

j 

:  \ 
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6.  RECOMMENDATIONS  FOR  MONTEREY  III 

The  Software  Documentation  Panel  should  be  recalled  to  address 
the  following  issues: 

1.  Review  MIL-STD-XXX 

2.  Review  the  Guidebooks 

3.  Examine  evolving  technology  and  methodologies 
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APPENDIX  C  -  DRAFT  SOW  TO  IMPLEMENT  A  TASK  TO  DEVELOP 
A  GUIDEBOOK  FOR  DOCUMENT  SELECTION 


1.  BACKGROUND 

The  Acquisition  Manager  (AM)  has  always  had  a  difficult  task  deciding 
the  amount  of  documentation  required  by  a  specific  project.  The  selection 
of  the  required  software/firmware  documentation  is  extremely  difficult 
(complex)  due  to  such  factors  as  system  complexity,  project  resources, 
development  stage,  documentation  overlap,  etc.  In  the  past,  the  AM  has 
typically  required  too  much,  too  little,  or  the  wrong  documentation 
because  he/she  had  little  or  no  basis  for  document  selection. 

The  task,  therefore,  is  to  develop  a  guidebook  which  will  select 
the  minimum  set  of  software/ firmware  documentation  based  upon  project 
characteristics.  This  guidebook  is  applicable  for  Embedded  Computer 
System  (ECS)  Sof tware/ Firmware  (on-line  and  support  software). 

The  total  effort/ task  shall  be  broken  up  into  three  subtasks: 

a.  Generate  the  DID  selection  algorithm 

b.  Validate  the  algorithm  generated 

c.  Write  the  DID  selection  guidebook 

The  subtasks  are  described  in  detail  in  Section  2  below. 

2.  TASK  DESCRIPTION 

a.  Algorithm  Development 

The  contractor  shall  develop  an  algorithm  which  takes  as 
input  various  characteristics  of  a  program  and  produces 
as  output  the  minimum  subset  of  software  DIDs  necessary 
for  the  proper  documentation  of  the  program. 

The  algorithm  may  take  any  form  that  the  contractor  deems 
appropriate;  however,  whatever  form  is  developed  shall  be 
concise  and  easy  to  use. 


C-l 


The  following  characteristics  shall  be  considered,  as 

a  minimum: 

1)  Will  the  program  require  interfacing  between 
multiple  services,  multiple  program  offices, 
multiple  contractors,  or  multiple  agencies 
(acquiring,  using,  maintaining)? 

2)  How  complex  is  the  software?  Will  multiple 
processors  be  used?  What  size  is  the  software? 

Will  there  be  timing  constraints?  Is  the  tech¬ 
nology  being  used  already  state-of-the-art,  or 
beyond  it? 

3)  How  critical  is  the  software  to  the  correct  and 
safe  functioning  of  the  system? 

A)  What  significance  does  software  have  in  the  overall 
system?  Is  it  hardware,  intensive,  software  intensive, 
operator  intensive? 

5)  What  are  the  interoperability  requirements  of  the 
system? 

6)  Fcjr  what  application  will  the  software  be  used,  i.e., 

C  ,  avionics,  fire  control,  flight  control,  etc.? 

7)  Will  the  software  requirements  be  firm  during  the 
life-cycle  of  the  system?  Is  an  upgrade  of  the 
system  planned? 

8)  What  is  the  projected  lifetime  of  the  software?  Is 
it  planned  to  be  replaced  in  the  near  future?  Is 
this  program  a  fast,  short  term  solution  which  will 
be  replaced  by  a  long  term  solution  when  it  is 
completed? 

9)  What  are  the  resource  constraints?  Is  there  sufficient 
manpower  to  manage  and  review  the  documentation?  Is 
there  enough  money?  Is  the  program  manager  working 
with  a  compressed  schedule? 

10)  Is  there  existing  software  code  (either  off-the-shelf  or 
from  a  previous  program)  which  can  be  used? 

11)  Is  there  existing  documentation  which  can  be  used? 


12)  Is  the  same  contractor  being  used  for  different 
phases  in  the  system  acquisition  life-cycle?  Can 
his  software  development  plan,  configuration  manage¬ 
ment  plan,  etc.  from  the  last  phase  be  used  for  this 
phase? 


13)  What  are  the  anticipated  maintenance  requirements? 

14)  In  what  phase  of  the  acquisition  life-cycle  is  the 
program? 

b.  Validation  of  DID  Selection  Algorithm 

The  algorithm  and  its  documentation  shall  be  validated  for 
correctness  in  selecting  the  critical  subset  of  DIDs  for  a 
given  ECS  sof tware /firmware  development  program  and  for 
usability  by  project  manager's  staff  in  clearly  and  directly 
guiding  him  in  DID  selection.  The  following  shall  be  con¬ 
sidered  for  validation  methodology: 

1)  Coordinated  review  by  a  representative  sample  of 
Government  and  Industry  software  development  and 
acquisition  specialists  and  project  manager's  staff. 

2)  Analyses  of  past  ECS  developments  comparing  the 
results  of  the  application  of  the  DID  selection 
algorithm  to  the  actual  DIDs  used  in  those  develop¬ 
ments  and  their  effectiveness. 

3)  Selection  of  pilot  case  developments  to  use  the 
DIDs  selected  by  the  algorithm.  Data  shall  be 
collected  and  evaluated  on  algorithm  usability 
and  effectiveness  of  selected  DIDs. 

c •  Development  of  the  DID  Selection  Guidebook 

The  Contractor  shall  prepare  a  DID  selection  guidebook  to  be  used 
by  program  managers  in  the  selection  of  DIDs /Documents  in  the 
acquisition  of  Embedded  Computer  System  (ECS)  Sof tware/Firmware . 
The  following  items  are  to  be  included  to  aid  the  Contractor 
in  his  thought  processes  and  provide  guidelines.  (These  data 
are  meant  to  be  a  minimum  and  may  be  expanded  upon  by  the  JLC.) 

1)  The  guidebook  shall  include  a  tutorial  on  why 
Sof tware/Firmware  documentation  is  needed, 
including  the  following  topics: 

-  maintenance  of  sof tware /firmware 

-  high  personnel  turnover 

-  planning  and  good  design 

.  -  agreement  that  design  is  acceptable 

'  -  management/development  overviews. 

The  Contractor  shall  include  case  histories  to  reinforce 
the  need  for  documentation. 

2)  The  guidebook  shall  include  a  section  which  identifies  all 
pertinent  military  standards  (e.g.,  1679,  483,  881,  490,  XXX, 
etc.)  effecting  the  use  of  DIDs  and  any  appropriate  comments 
on  their  applicability  or  use. 
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3)  The  Contractor  shall  study  the  DIDs  and  DID  overviews 
(attached)  and  modify  the  DID  overviews  to  state  when 
each  particular  DID  is  needed  or  not  needed.  The 
expanded  'DID  overviews"  shall  be  included  in  the 
guidebook. 

4)  The  use  of  the  algorithm  or  selection  process  shall 

be  demonstrated  in  the  guidebook  with  examples /ins trac¬ 
tions  on  its  effective  use. 


APPENDIX  D  ADDITIONAL  DOCUMENTS 


D.l  FIRMWARE  SUPPORT  MANUAL 


The  Firmware  Support  Manual  provides  Information  necessary  to  modify 
the  Read-Only-Memory  (ROM)  components  of  an  embedded  computer  system. 
Specifically,  the  Firmware  Support  Manual  describes  the  physical  ROM 
components,  installation  and  repair  procedures,  ROM  programml no/ testing 
equipment  ana  procedures  for  ROM  "burning,"  and  any  special  handling 
or  security  requirements.  In  the  above  discussion,  the  term  "ROM" 
encompasses  ROM’s,  PROM's,  EPROM’s,  UVPROM's,  and  other  forms  of  ROM 
devices. 

The  Firmware  Support  Manual  is  not  intended  to  provide  information 
regarding  the  design  and  implementation  of  the  computer  programs 
represented  by  the  bit  pattern  within  the  ROM.  These  computer  pro¬ 
grams  are  incorporated  into  the  mainstream  of  software  development 
(and  documented}  just  as  any  computer  program  module  intended  to 
reside  on  read-write  memory. 

Relevant  DIOs  are: 

1.  USAF  UDI-E-3937-AS0  "Firmware  Support  Data." 

2.  NSA/CSS  DI-H-5526  "Firmware  Data  Abstract  and  Document 
Record. 

3.  NSA  U-H-5552  "Firmware  Documentation  Standards." 

4.  EIA  draft  "Hardware  Intensive  Firmware  Description. 


0-1 


0.2  OPERATIONAL  CONCEPT  DOCUMENT 


The  Operational  Concept  Document  provides  explicit  description  of 
the  way  in  which  the  data  processing  system  will  appear  to  its  users 
and  the  way  in  which  't  will  interact  with  them.  It  describes  assumption 
being  made  about  how  tne  user  will  use  the  system  to  perform  his 
operational  tasks.  It  is  derived  from:  System  Operational  Concept, 
System  Specification,  C*  and  CPCI  identification,  knowledge  of  current 
operations  and  environment,  etc.  Although  the  system  is  typically  In 
an  early  stage  of  development  and  the  exact  production  baseline  con¬ 
figuration  Is  unknown  when  the  OCD  is  first  written,  this  document 
serves  to  clarify  and  make  explicit  the  intended  appearance  and 
function*!  capabilities  of  ths  future  system.  The  OCD  provides  a  basis 
for  concensus  and  understanding  between  tne  various  agencies  and 
contractors  involved  and  serves  as  a  valuable  source  of  information 
during  design,  implementation,  test  and  acceptance  of  the  system.  The 
CCD  is  delivered  prior  to  the  System  Design  Review  (SDR)  and  the  approved 
Operational  Concept  is  formally  presented  curing  the  SDR. 
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REQUIREMENTS  TRACEABILITY  MATRIX 


The  Requirements  Traceability  Matrix  (RIM)  explicitly  Identifies  each 
requirement  specified  In  the  System/Segment  Specification  and  provides 
a  mechanism  for  tracing  the  allocation  of  these  requirements  to  the 
hardware  and  software  components  of  the  system/seqment  (e.g., 
MIL-STD-483/490  Cl 's  and  CPCI's).  In  the  case  where  a  system  is 
divided  into  segments,  the  RTM  documents  the  allocation  of  system  re¬ 
quirements  to  segments,  and  then  to  the  lower  level  specifications 
and  documents:  Software  Requirements  Specifications,  Software  Prelim¬ 
inary  and  Detailed  Design  Documents,  Software  Product  Specifications, 
and  the  appropriate  hardware  specifications  and  documents.  The  RTM, 
as  it  evolves,  provides  a  basis  for  testing  and  acceptance. 

Ideas  to  be  contained  in  DID  block  10  are  given  on  the  following  page 


PROPOSED  DIO  BLOCK  10 


10.  Preparation  Instructions  for  Requirements  Traceability  Matrix 

The  content  of  the  Requirements  Traceability  Matrix 
shall  be  as  follows:  The  leftmost  columns  of  the  matrix 
shall  contain  paragraph  and/or  subparagraphs  numbers  of  each 
system/system  segment  (or  top-level)  specification 
requirement  and  the  test  method (s)  of  that  requirement  from 
the  test  verification  matrix.  Adjacent  column  sets  shall 
show  the  allocation  of  requirements  to  lower  level 
specifications  by  listing  corresponding  paragraph  and/or 
subparagraph  numbers  in  those  documents  and  shall  show  the 
test  method  of  each  allocated  requirement  from  each 
document's  test  verification  matrix.  For  traceability  of 
requirements  between  the  system  specification  and  the 
segment  specifications,  there  shall  be  a  separate  column  for 
each  segment  specification.  For  traceability  of  requirements 
between  a  system  or  segment  specification  and  lower  level 
specifications,  there  shall  be  separate  columns  sets  for 
each  development  (type  B) ,  product  (type  C) ,  and  interface 
specification  (IFS)  or  interface  control  document  (ICD). 

Note  that  test  methods  for  C  specs,  IFS's  and  ICD's  may  not 
be  applicable.  Submatrices  which  repeat  the  development 
specification  requirements  in  the  left  column  set  and  the 
corresponding  product  specification  paragraph (s)  in  the 
right  column  set  are  acceptable.  Initial  submittal  of  the 
RTM  shall  include  the  available  segment  information  as  well 
as  interface  specifications.  Updates  shall  accompany 
subsequent  new  specifications  submittals,  revisions  to 
approved  specification,  (e.g.,  via  SCN's),  and  resubmittals 
of  draft  specifications  which  contain  changes  in  their 
paragraph  numbering  relative  to  higher  level 
specifications.  Revisions  to  the  RTM  in  response  to 
comments  made  by  the  SPO  shall  be  submitted  by  the  contractor 
within  fifteen  (15)  working  days  of  receipt  of  comments. 


3.x  Requirements 

Develop  a  means  for  requirements  traceability  which  will  indicate  where, 
in  each  development,  product  and  interface  specification  (or  ICD),  each 
of  the  requirements  in  the  system/system  segment  (type  A)  specification 
is  addressed  and  which  will  indicate  the  corresponding  test  method  for 
each  of  those  requirements. 
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APPENDIX  E 


COMPUTER  RELATED  DATA  ITEM  DESCRIPTIONS 
NOT  ADDRESSED  IN  THE  JLC  DID  STUDY  DATED  1  JUNE  1981 


SOURCE 


DID  NUMBER 

TITLE 

DOCUMENT  NO. 

DI-H-3277 

Training  Equipment  Computer  Pro¬ 
gram  Documentation 

MIL-STD-876 

UDI-I-E-3935 

Firmware  Development  Plan  (FDP) 

AFR  800-14 

UDI-E-3936-ASD 

Firmware  Technical  Description 
(Product  Specification) 

MIL-D-83468 

UDI-E-3937-ASD 

Firmware  Support  Data 

AFR  800-14 

DI-M-5085 

Commercial  Computer  and  Peripheral 
Equipment  Manuals 

UDI-H-5503 

Firmware  Reprocurement  Specifi¬ 
cation 

DI-H-5526 

Firmware  Data  Abstract  and 

Document  Record 

UDI-H-5552 

Firmware  Document  Standards 

MIL-STD-100 

DI -A- 30001 

Software  Delivery  Documentation 

DI-A-30023 

ADPE  Systems  Configuration 

Report  (Schematics) 

DI-E-301 02 

Computer  Program  Detail 

Specification 

DI-E-301 12 

Computer  Program  Listings 

DI-E-30141 

Interface  Specification 

MIL-STD-483 

DI-E-301 49 

Research  &  Development 

Computer  Software 

DI-E-301 50 

Visual  Data  Base  Description 

Document 

MIL-STD-490 

DI-F-30202 

Automatic  Data  Processing 
(ADP)  Manpower  and  Cost 

AFM-171-404 

APPENDIX  E  CONTINUED 


SOURCE 


DID  NUMBER 

TITLE 

DOCUMENT  NO. 

DI-L-30309 

Automatic  Data  Processing  Equip¬ 
ment  (ADPE)  Cost  and  Utilization 

AFM-300-6 

DI-M-30404 

Computer  Program  Configuration 

Item  Users  Manual  (Operational 
Software) 

AFSCM/AFLCM 
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DI-M-30405 

Computer  Programing  Manual 

DI-M-30408 

Computer  Program  Configuration 

Item  Maintenance  Manual 
(Operational  Software) 

DI-M- 30410 

Computer  System  Operational 

Manual 

APPENDIX  F  DID  COMMENTS 


Table  I  of  the  JLC  Report  (reference  1),  page  5  lists  29  documents 
reconmended  by  the  CSM  Subgroup.  Of  these,  new  DIDs  were  developed  for 
23.  The  DID  contractor  recommended  that  existing  DIDs  be  used  for  the 
other  six. 

The  Documentation  Panel  reviewed  the  23  new  DIDs  plus  four  of  the 


existing  DIDs.  Comments  on  these  are  on  the  following  pages.  We  ignored  ,| 

the  Engineering  Change  Proposal  and  the  Specification  Change  Notice.  | 


R-DID- 101 /SYSTEM/SEGMENT  SPEC 


Below  are  comments  on  the  software  aspects  and  on  the  general  aspects 
of  the  System/ Segment  Spec  DID.  A  considerable  number  of  changes  and 
improvements  are  recommended.  Problem  areas  are  also  noted  such  as  the 
level  of  detail  called  for  on  the  interfaces  (3.1.5).  Their  resolution 
will  impact  the  life  cycle  model  as  depicted  In  Figures  1  and  2  of  the  DID 
package. 


1.  Spell  out  all  abbreviations  (e.g.,  DMA,  ISA  etc.). 

2.  Move  Configuration  Identiflcatlon/Spec  tree  from  3.1  to  3.3.1. 

3.  Add  Interface  Requirements  to  Spec  tree. 


4. 

3.1.3 

After  "shall  contain"  add  "as  a  minimum." 

5. 

3. 1.5.1 

3. 1.5. 2 

After  "quantitive  terms"  add 

3. 1.5. 3 

"such  as." 

6. 

3.2.2 

Delete  g.  "comnand  and  control  requirements. 

7. 

3.3 

(1)  Insert  "and  computer  software"  after 

system  equipment." 


(2)  Don't  understand  last  sentence  -  What 
does  "specify  criteria"  mean? 

8.  3.3.1  This  section  should  state  then  an  appendix 

may  be  used  for  the  specification  tree. 

The  example  figure  should  be  updated  to 
include  an  Interface  Specification. 

9.  3. 3. 2. 3  Computer  System  Requirements 

Add  "existing  hardware  and  software  items" 
Delete  "etc." 


10.  Add  a  paragraph  (3. 3. 2. 4)  on  Computer  Resources  Utilization 
Measurement.  This  paragraph  shall  specify  computer  resources 
utilization  monitoring  requirements.  Included  shall  be  the 
requirement  that  resources  be  dynamically  monitored  during  real 
time  operations,  the  variability  of  data  recording  intervals,  the 
functions  under  operator  control,  and  the  methods  of  displaying/ 
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reporting  the  utilization  data. 

11.  Add  paragraph  (3. 3. 2. 5)  on  Firmware.  This  paragraph  shall  state 
the  requirements  for  programming  languages  (3. 3. 2.1)  and  design 
and  coding  standards  (3. 3.2.2)  which  apply  to  firmware  as  well 
as  software. 

12.  The  subsections  on  maintenance  and  supply  should  be  expanded 

to  include  software.  It  is  important  to  provide  guidance  as  to 
where  software  maintenance/modification  will  be  done  and  how 
software  upgrades  will  be  distributed  to  the  field. 

13.  System  Spec/SW 

3. 4. 3. 2  Computer  Resources  Support 
Change  first  sentence  to: 

"This  paragraph  shall  specify  the  computer  resources  facilities, 
equipment,  and  software  to  be  provided  for  CSCI  maintenance 
during  the  system's  operational  service  life." 

14.  Section  4  should  be  titled  "Test  Requirements."  There  seems 
to  be  considerable  overlap  between  the  information  asked  for 
here  and  in  the  Test  Plan. 

15.  System/ Segment  level  verification  cross  reference  matrix  should 
be  added  to  section  4. 

16.  Add  a  glossary  in  Section  6. 
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R-DID-1 04  SOFTWARE  DEVELOPMENT  PLAN 


1.  1.2  -  Expand  somewhat  to  describe  generai  Implementation 
approach  e.g.,  versions,  languages,  tools).  This  also 
might  be  the  place  to  discuss  categories  of  software  and 
the  level  of  attention  to  be  applied  to  each  category  (e.g., 
how  will  operating  systems  from  hardware  vendors  be  treated?) 

2.  6.1  -  typo  -  "computational." 

3.  Section  6.5  "Preparation  for  Delivery"  is  in  System  Spec 
section  5,  (DID  101). 

4.  Paragraph  7  Should  include  techniques  for  managing  the 

allocation  of  development  resources, 
priorities  who  resolves  conflicts  etc. 

5.  7.1  Add  firmware  -  unique  facility. 

6.  Paragraph  9.0  Development  plan  would  be  a  logical  place  to 

document  developer's  understanding  of  what 
criteria  must  be  fulfilled  to  satisfy  each 
program  milestone.  For  example: 

What  constitutes  data  package  for  PDR?  What 
documentation  in  package?  To  what  level  of 
detail?  Whose  approval  must  be  obtained  to 
hold  PDR  meeting?  Can  PDR's  be  segmented?  etc. 

7.  Add  the  following  items  from  existing  DID  30567A: 

system  support 
proprietary  items 
long  lead  items 
111  ties  treatment 
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R-DID-105  SOFTWARE  CONFIGURATION  MANAGEMENT  PLAN 


1.  This  DID  does  not  discuss  or  reference  the  concept  of  "incremental 
builds"  or  partial  deliveries  and  the  impact  on  software  con¬ 
figuration  control . 


2.  Add  a  section  dealing  with  implementation  similar  to  section  6  of 
Software  Quality  Assurance  Plan,  R-DID-106. 

3.  1.1  Add  abbreviation  to  title  "(SCMP)." 

4.  1.3  Replace  "the  approval  channel  for  making  changes."  to 

"controlled" 

5.  4.1.d  What  does  "Contractual  Significance"  imply?  It  is  not 

a  standard  DOD  term. 


6.  9  Delete  reference  to  other  DIDs. 


MINORITY  REPORT 

This  plan  should  be  incorporated  into  the  overall  configuration 
management  plan  to  insure  that  configuration  management  is  consistent, 
that  the  impact  of  changes  on. all  parts  of  the  system  are  covered. 

This  can  only  occur  if  the  handling  of  all  ECPs  are  coordinated.  The 
information  in  this  DID  should  be  added  to  that  for  the  overall  CMP. 
Software  CM  personnel  should  help  develop  a  new  CMP  DID. 
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R-DID-106  SOFTWARE  QUALITY  ASSURANCE  PLAN 

1.  General  Comnents:  The  DID  should  be  rewritten  to  reflect 
changes  in  MIL-S-52779A. 

2.  9  Add  MIL-S-52279  and  MIL-S-5227A  and  MIL-STD-XXX. 

3.  10.4.1  Add  to  last  sentence  "and  test  standards." 

4.  4.3.4  Change  title  to  "Critical  Resource  Factors  Montoring." 

5.  4.5  Add  to  section  4.5 

e.  Method  of  documentation  of  problem,  corrective 
action,  and  test  results. 

6.  Add  a  new  section.  4.6  Controls  for  Computer  Program  Library. 

7.  Add  a  new  section. 

4.7  Work  Certification.  Describe  the  procedures  and  controls  for 
the  handling  of  source  coda  and  object  code  and  related  data  in 
their  various  forms  and  versions,  from  the  time  of  their  initial 
approval  or  acceptance  until  they  have  been  incorporated  into  the 
final  media.  This  shall  include  controls  to  ensure  that  different 
computer  program  versions  are  accurately  identified  and  docu¬ 
mented,  that  no  authorized  modifications  are  made,  that  all 
any  ,<ed  modifications  are  properly  incorporated,  and  that  soft¬ 
ware  submitted  for  testing  is  the  correct  version. 

8.  Block  7  "...internal  plan  or  agreements..."  Imply  that  H  is  not 
a  deliverable  data  item.  Reword  "It  documents  the  contractor's 
quality  assurance  program  as  applied  and,  if  necessary,  or 
tailored  to  the  specific  contract." 

9.  4.3.1  thru  This  secw.on  might  be  the  data  that  should  be 

provided  as  Section  5.6  of  the  CRl.CMP,  Volume  1, 
as  it  relates  to  a  management  function.  These 
paragraphs  Infer  that  the  QA  process  is  to  perform 
-jll  the  listed  activities.  The  QA  process  is  to 
assure  they  happen,  and  in  accord  with  contract 
promises  or  company  practices.  Change  context. 


10. 


Section  5  of  MIL-S-52779  Preparation  for 
Delivery,  has  been  deleted.  We  suggest  that 
Section  5.6. 3.1  of  the  CRLCMP,  Volume  1,  ■;» 
placed  in  the  SQAP  and  Sections  5. 6. 3. 2  arid 
5.6. 3.3  be  part  of  the  SCMP  which  are  then 
monitored  for  compliance  by  the  SQA  group. 

11.  Blocks  3  &  /  Add  "or  activity"  wherever  "contractor"  is 

stated. 
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R-0I0-107  SOFTWARE  PROBLEM/ CHANGE  REPORT 

DELETE  THIS  DID 


R-DID-108  SOFTWARE  REQUIREMENTS  SPECIFICATION 


1.  Rev  rite  DIO  to  use  483  Notice  2. 

2.  A  subsection  (or  subparagraph)  should  be  added  to  the  D!n  wherein 
the  computer  resources  requirements  allocated  to  the  particular 
CSCI  are  to  be  placed.  This  includes  programming  languages, 
design  coding  and  standards,  and  computer  system  requirements  of 
the  System/Segment  Spec.  (3.3.2). 

3.  Although  OlD's  can  include  an  example  when  implemented  for  a 
specific  system,  the  OID  itself  should  not  have  general  examples 
because  they  are  misleading,  e.g.  FIGURE  2  has  the  design  in 
Software  Specification  instead  of  concentrating  on  requirements. 

4.  Include  definition  of  Inspection,  Analysis,  Demonstration  and 
Test  in  MIL-STD-XXX.  Add  to  Table  III  in  this  DID. 

5.  3&7  Delete  the  word  "digital"  from  the  phrase 

"digital  computer  software." 

6.  3.1.1  Interface  Requirements.  The  word  "contractor" 

should  be  deleted  from  the  4th  line.  The 
government  also  performs  analysis. 

Add  "whenever  separate  interface  specifications 
are  to  be  used  they  will  be  referenced  ir  this 
section. " 

7.  3.2  line  Delete  "and  followed  ...  Table  II" 

5  t0  7  Delete  Table  II. 

8.  3.2.17  The  traditional  B-5  format  for  functional 

requirements  specification  does  not  enhance 
the  software  engineering  process.  In  fact,  the 
use  of  separate  input  and  output  sections  leads 
to  confusion  and  conflicting  definitions  of 
the  function-to-function  interface. 

Recommend  that  the  traditional  format  be 
dropped  in  favor  of  a  format  that  requires 
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9.  3.3 


10.  3.4 


n.  4.o 


12.  Figure  2 


separate  sections  be  created  consisting  of 
specific  functicn-to-function  interface  defini¬ 
tion  subsections  and  a  separate  section  created 
consisting  of  specific  functional  processing 
description  subsections. 

Add  "whenever  a  separate  specification  for  data 
base  is  written  It  will  be  referenced  In  this 
paragraph." 

Delete  this  paragraph. 

Change  title  to  "Test  Requirements." 

Delete  figure  and  all  references  to  It. 

MINORITY  REPOPT 


There  should  always  be  an  IRS  if  a  CSCI  Interface,  with  another 
CSCI.  Then  there  will  only  be  one  source  for  all  design  groups  to  use 
for  Interface  details  and  there  will  only  be  one  document  to  change 
when  changes  are  made  to  an  Interface.  For  that  reason  paragraph  3.1.1 
should  be  changed  so  that  it  only  Identifies  the  interfaces  and 
references  the  IRSs.  Also,  the  material  in  3.1.2  3.1.3,  3.1.4  should 
be  deleted  and  the  appropriate  parts  therein  incorporated  into  the 
IRS. 


DIO  109  -  INTERFACE  REQUIREMENTS  SPECIFICATION 


1.  Block  3  delete  Last  sentence 

2.  Block  7,  line  7,  add  after  "contractors"  "or  activities"  since 
government  activities  sometimes  develop  this  document  in-house. 

3.  Block  7,  delete  second  paragraph. 

4.  The  IRS  does  specify  the  physical  portion  of  the  interface. 

The  DID  does  not  presently  call  for  that.  Contractors  on  each 
side  of  an  interface  must  have  the  electrical  characteristics 
(signals)  of  the  interface  as  well  as  the  functional  character¬ 
istics  (messages  and  data  elements)  defined.  This  must  be 
addressed. 

5.  Correct  punctuation  everywhere. 

6.  1.1  Is  interface  a  configuration  item?  and  does  it 

have  configuration  item  numbers  and  nomenclature 
as  the  DID  calls  for? 

7.  1.1  line  5  Delete  quotation  mark  before  "The  nomenclaure" . . . 

o  y 

and  after  ...  numbers)." 

U.  1.2  Should  be  made  to  read  plural  (i.e.,  interfaces,  .. 

their  relationship  ....  which  they  apply.) 

9.  3.1  Delete  the  underline. 

10.  Paragraph  3.2,  line  2,  after  "interfacing  equipment"  insert  "man 
to  machine  interface." 

11.  Paragraph  3.2,  line  3,  change  "computer  programs"  to  read  "the 
software. " 

12.  line  4  Delete  "where  applicable."  Phases  such  as  these 

only  lead  to  arguments  between  contractors  and 
customers  -  give  the  government  representative 
credit  for  recognizing  when  somethings  are 
legitimately  omitted  from  a  document. 

13.  3.2.1  Change  "Interface  Signal  Cross  Index"  in  title  to 

"Interface  Data." 
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Delete  "If  applicable. 


14.  3.2.1 
Line  2 

15.  3.2.1 . X . 1  Delete  quotation  marks. 

3.2.1 . Y . 1 

16.  3. 2.I.X. 5  Second  line  -  last  word  -  "related",  question- 

3.2.  l.Y.  5  "realted  to  what?" 


17.  3. 2.I.X. 14  Replace  "with  its"  with  "and." 

3. 2. l.Y. 14 


18.  Paragraph  3.2.1.X.4  -  The  following  sentence  appears:  "Each 
interface  data  requirement  shall  be  realted  to  its  corresponding 
Executive  Logic  requirements."  Again,  this  requirement,  makes 
no  sense.  What  is  required  within  this  paragraph,  with  regards 
to  content,  is  vague.  This  paragraph  should  be  rewritten  to 
make  clear  that  interlock  logic  is  being  addressed. 

19.  Paragraph  3.2.1.X.7  -  This  paragraph  is  incomplete.  The  para¬ 
graph  only  requires  that  the  "protocol  for  blocking,  message 
switching,  and  handshaking"  be  stated.  Error  conditions  and 
interface  testing  techniques  or  requirements  with  specific 
prose  pertaining  to  the  periodicity  of  tests  and  the  type  of 
data  to  be  tested  should  be  included. 


20.  3. 2. 1.x. 15  Add  a  subsection  calling  for  limits  and/or  ranges 

of  values  and  accuracy/precision  requirements. 

Or  change  3. 2. 1.x. 11  title  to  Quantitative  Des¬ 
cription  of  Data  Elements"  and  combine  material 
here  with  units  of  measure. 

21.  3.2.2(all)  The  specific  comments  on  paragraphs  3. 2.1. all 

apply  to  like  paragraphs  of  3.2.2. 


22.  Paragraph  6.  Delete  last  sentence. 

23.  4.  Delete  entire  paragraph. 

24.  Delete  sections  5,  7,  8  &  9. 


M3 


MINORITY  REPORT 


3.2.1 . x. 2 

3.2.1 . x. 4 

3.2.1 . x. 6 

3.2.1 . x9 

3.1.1 . x. 12 


(1)  The  requirements  for  shared  memory,  executive 
logic  control,  interrupts,  memory  map,  and  shared 
variables  apply  only  to  interface  between  two  or 
more  CSCIs  that  share  the  same  computer.  They 
should  be  categorized  as  such  and  grouped  together. 

(2)  The  material  called  for  in  these  paragraphs  is 
at  the  design  level  not  the  requirements  level  and 
will  not  be  known  at  the  time  the  IRS  is  due,  which 
is  between  SDR  and  PDR.  It  should  be  in  the  IDD. 


M4 


R-DIO-llO  SOFTWARE  STANDARDS  AND  PROCEDURES  MANUAL  (SSPM) 


1.  Delete  Section  3.  It  is  an  example  of  a  paragraph  that  in¬ 
creases  the  cost  of  a  document  without  adding  utility.  It  calls 
for  material  (guidelines,  philosophy,  or  rationale  used  in  de¬ 
fining  the  software  performance  requirements,  and  in  allocating 
requirements  to  design  elements)  which  can  be  obtained  elsewhere 
and  which  is  volatile  in  nature.  Philosophy  and  rationale  will 
change  as  the  design  evolves  and  as  ECPs  are  made  and  therefore 
the  text  will  become  outdated  unless  the  SSPM  is  changed.  It 
probably  won't  be.  Further,  it's  good  to  avoid  situations  where 
many  documents  have  to  be  changed  for  a  change  to  the  CSCI(s). 


2. 


3. 


4.4.1 

thru 

4.4.3 

5.4 


4.  6. 1,6, 2, 

and  6.3 


5.  6.4.2 


6.  6 


Delete  these  paragraphs. 


•  This  paragraph  should  Include  a  discussion  of  when 
it  will  be  permissible  to  use  assembly  language. 

Delete  the  sentence  in  each  of  these  sections, 
which  defines  the  purpose  of  the  activity. 

Change  the  last  sentence;  for  example  to  "Define 
the  integration  testing  methodology,  top-down, 
bottom-up  etc.". 

This  information  should  be  included  in  the  QA  Plan, 
except  for  section  6.6  which  should  be  included  in 
the  Software  Development  Plan. 
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DID  111  -  SOFTWARE  TOP  LEVEL  DESIGN  DOCUMENT 


1 . 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 


Paragraph  numbers  of  DIDs  111  and  112  snould  mutch  to  allow 
traceability  between  the  two  levels  of  design;  example:  Oata 
base  is  Section  3.S  in  this  DID  but  is  Section  3,3  in  DID  112. 

All  acronyms  and  abbreviations  should  be  official  JLC  and 
should  be  found  in  a  single  glossary. 

Frequent  use  of  "may'1  vice  "shall"  in  r.O'itractual  statements. 

Numerous  typo's  and  incorrect  use  of  capitalization  should  be 
corrected.  Also,  correct  punctuation. 

Slock  7:  Change  "baselined"  to  "reviewed."  Change  ‘ SE&TD ,  IV&V 
to  "t.he  GSE&l  and  IV&V  Contractors..." 


1.1  CSC  I ' s  should  be  allowed  to  be  plural,  since  it 

is  possible  that  one  software  Top  Level  Design 
Document  could  be  the  ba=>is  for  morf.  than  one 
program  (CSCl). 

1.1  Third  line,  lead' line  with  'identified  as  ..." 

3.2  Need  to  have  a  section  that  addresses  each  of  the 
CSCi's  and  indicate  which  CSC's  apply  to  which 

CSC  I  (i.e.,  module  to  program  allocation).  Replace 
Figure  1  with  the  attached. 

3.3  Review  and  rewrite  sections.  There  is  significant 
difference  between  information  and  control  flow 
neither  of  which  are  adequately  covered. 

3.3.2  There  should  be  a  requirement  to  specify  worst 

spd 

3  '3  4  case  timing  roqui rements .  What  functions  muct 

be  executable  simultaneously,  etc.  Timing  for 
individual  CSCs  does  not  give  a  complete  picture 
in  a  multiprogramming/multitaskir.g  environment. 

3.4  The  N2  chart  is  subject  to  controversy  in  regard 
to  its  utility.  Delete  the  N12  chart  (figure  4) 
and  all  references  to  it  in  the  paragraph. 
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12. 


Paragraph  3.4. X,  first,  line,  the  word  "intent"  is  redundant  when 
combined  with  purpose;  delete  "intent  and." 

13.  Add  a  new  paragraph,  3.4.x. 5,  to  address  error  detection  and 
recovery. 

14.  Need  subsection  (3.7)  for  CPU  allocation  to  cover  multiprocessing 
and  multiprogramming  applications. 

15.  Delete  paragraphs  4,  5,  7,  3,  9. 

Requirements 


CSC1  CSC2  .  CSCH 


3.2.1 

X 

...  . 

3.2.2 

X 

3.2.n 

• 

• 

X 

• 

X 

3.2.m 

j 

i 

X 

FIGURE  1  REPLACEMENT 


MINORITY  REPORT 

The  Software  Top  Level  Design  Document  and  Software  Detailed 
Design  Document  should  be  combined  to  form  the  Software  product 
spec,  eliminating  2  DIDs.  This  document  could  be  delive  'ed  in 
snapshot  forms,  as  required,  until  the  entire  document  is  accepted. 
The  Configuration  Management  and  delivery  problems  with  this  scheme 
are  what  should  be  addressed. 
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DID  112  -  SOFTWARE  DETAILED  DESIGN  DOCUMENT 


1.  Paragraph  numbers  of  DID  112  should  match  DID  111  as  much  as 
possible  to  allow  traceability  and  minimize  confusion. 

2.  General  -  recommend  that  an  optional  section  for  programming 
guidelines  be  Included  to  allow  a  small  project  to  reduce  the 
numbers  of  required  documents. 

3.  Correct  punctuation. 

4.  Paragraph  1.1,  Need  to  allow  CSCI's  to  be  plural,  since  It  is 
possible  that  one  Software  Detailed  Design  Document  could  be 
the  basis  for  more  than  one  program  (CSCI). 

5.  Paragraph  3.1,  Need  to  address  each  of  the  CSCI's  and  INDICATE 
which  CSC's  apply  to  which  CSCI  ( 1 . e . ,  module  to  program 
allocation) . 

6.  Paragraph  3.1.1,  Should  be  3.I.X.  When  referring  to  routines,  the 
term  routines  needs  to  be  defined  -  some  systems  address  them  as 
subroutines,  some  as  procedures.  Believe  this  should  be  address¬ 
ed  as  the  lowest  level  program  structure  element. 

7.  Paragraph  3. 1.1.1  should  contain  an  example  of  a  chart  or  table 
illustrating  functional  allocation. 

8.  Paragraph  3. 1.1. 2  should  probably  include  an  update  to  Table  I  of 
the  DID  111 . 

9.  Paragraph  3. 1.1. 3. Y  -  all  bullets  should  be  numbered  sub-para¬ 
graphs. 

10.  Paragraph  3.1.1.Y  Care  must  be  taken  to  make  certain  we  don't 
just  verbalize  code.  This  area  must  be  kept  at  a  level  above 
coding.  We  have  gotten  into  the  trap  of  verbalizing  code  in 
the  post  and  the  document  maintenance  becomes  overwhelming. 

11.  Paragraph  3. 1.1. 3. Y,  first  bullet,  change  "development"  to  read 
"description." 

12.  Paragraph  3.3,  fourth  line,  reconmendations  to  delete  "so  as  to 
require  a  data  base  manager." 
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13.  Oelete  paragraphs  4,  5,  7,  8,  9. 


MINORITY  REPORT 

The  Software  Top  Level  Design  Document  and  Software  Detailed 
Design  Document  should  be  combined  to  form  the  Software  product 
spec,  eliminating  2  DIDs.  This  document  could  be  delivered  In  snap- 
shop  forms,  as  required,  until  the  entire  document  Is  accepted.  The 
Configuration  Management  and  delivery  problems  with  this  scheme  are 
what  should  be  addressed. 


F-19 


DID  113  -  INTERFACE  DESIGN  DOCUMENT 


1.  Recommend  that  the  title  be  changed  to  "Interface  Design 
Specification." 

2.  General,  Data  required  at  the  IRS  and  IDD  levels  which  are 
missing  and  should  be  added  are  error  condition  detection 
and  Interface  testing  requirements. 

3.  The  "model"  the  IDD  (and  the  IRS)  is  based  on  assumes  that 
Interfacing  CSCIs  will  all  execute  in  the  same  computer. 

Thus  the  IDD  DID  calls  for  things  (executive  control  logic, 
shared  variables)  that  will  not  apply  to  many  projects. 

Most  system  to  system  interfaces  will  not  have  CSCIs 
executing  in  the  same  computer.  The  DID  should  be 
changed  to  accommodate  both  situations. 

4.  Block  3  Relationship  of  the  IDD  to  the  IRS  should  be 

presented. 

5.  Block  7  Line  3,  delete  "too."  Between  "or"  and  "several" 

insert  "if." 

6.  3,1  Line  5,  delete  "it  is"  and  insert  "they  are". 

Delete  N^  chart  reference  and  Figure  1. 

7.  3. 2.I.X. 4  This  section  requires  "detailed  design  of 

executive  control  logic."  This  requirement 
makes  no  sense.  What  is  meant  by  "executive 
control  logic"  should  be  explained  or  the 
paragraph  deleted.  This  paragraph  is  undoubt¬ 
edly  intended  to  cover  methods  for  shared  data 
and  should  be  reworded  to  so  state. 

SPLIT  DECISION 

The  panel  was  equally  divided  on  accepting  or  rejecting  the 
following  comments. 

8.  1.1  Following  the  word  "between"  change  remainder 

of  sentence  to  "(insert  nomenclatures  and  con¬ 
figuration  item  numbers  of  interfacing  CSCIs 


F-20 


9.  1.2 


and  CIs)." 

Delete  this  subsection.  As  is,  direction  in 
DID  is  vague,  calling  for  "overall  interface 
design"  and  "major  design  issues."  Will  only 
add  to  cost  of  IDDs. 

10.  3.2.1  Change  3.2  to  a  title  only  paragraph  and  delete 

text  calling  for  brief  overview  and  discussion 
of  key  design  issues.  Only  adds  to  IDD's  cost. 

(1)  Change  "Signal"  to  "data  element"  here  and 
throughout  IDD.  Signals  are  the  hardware-to- 
hardware  realization  of  the  interface. 

11.  3.2.1.X.1  Delete  "and  give  an  overview  o*  the  key  design 

issues. . ." 

12.  Define  "design  considerations"  and  provide  details  on  what  is 
expected  from  the  contractor.  Actually  this  is  material  only 
appropriate  at  design  reviews  and,  if  the  customer  feels  the 
need,  documented  as  study  reports. 

MINORITY  REPORT 

13.  Block  7,  the  IDD  should  always  be  required  when  a  computer  co 
computer  interface  is  involved. 

14.  Block  7,  the  IDD  is  required  in  order  to  adequately  detail 
performance  requirements  in  the  SRS.  It  should  be  produced 
earlier  in  the  cycle  than  shown. 

15.  The  IDD  must  be  a  government  controlled  specification.  It  is 
both  controlled  and  maintained  by  the  government  and  changes 
can  only  be  made  via  FCP  once  it  is  baselined. 

16.  Block  7,  (1)  Message  presented  should  be  changed  to  always 
calling  for  an  IDD.  If  left  to  the  contractor,  pressure 
will  be  to  define  extremely  complex  information  as  being 
very  simple.  The  major  reason  there  should  always  be  a 
separate  IDD  is  the  avoidance  of  the  redundancy  of  repeating 
che  interface  material  in  two  or  more  Detailed  Design  Documents 
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and  to  decrease  the  potential  that  the  different  design  and 
coding  groups  will  be  working  from  different  versions  of  the 
interface.  Also,  only  the  100  will  have  to  be  changed  when 
changes  are  made  to  the  interface  design. 

(2)  Design  documents  should  be  controlled  by  the  contractor  - 
delete  last  sentence  starting  with  "It  is  controlled . " 


17.  Two  panel  members  disagree  with  comment  3. 


13.  3.1 


19.  3. 2. 1.x. 2 


20.  3. 2.I.X. 7 


(1)  How  does  the  block  diagram  called  for 
here  differ  from  that  called  for  in  the  IRS? 
Why  have  both  of  them? 

Changes  in  formats  should  be  clearly 
delineated  in  3.2.X.1  not  here.  Delete 
sentence  starting  with  "Indicate  whether..." 

Memory  Maps  are  not  required  to  detail  an 
interface.  Place  it  in  the  design.  Re¬ 
locatable  modules  and  segments  make  this 
an  impossible  task. 


21.  3.2.1.X3 

3. 2. 1. x. 4 

3. 2. 1. x. 5 

3. 2. 1. x. 6 

3. 2. 1. x. 7 

3. 2. 1. x. 10 


The  material  called  for  these  subsections 
apply  only  to  CSCIs  which  share  the  same 
computer  (CPU  and  memory).  A  qualification 
should  be  added  to  the  DID  which  makes  this 
explicit  -  for  those  CSCIs  which  do  not  share 
the  same  computer,  material  such  as  executive 
control  design,  priority  determination 
algorithms  belong  in  each  respective  CSCI 
Detailed  Design  Spec,) 
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R-DID-114/UATA  3ASE  DESIGN  DOCUMENT 


General  Comment 

DID-114,  as  currently  written,  was  found  inadequate  and  was 
rewritten.  The  rewritten  version  is  attached  under  the  title,  "A 
Working  Paper  on  DATA  BASE  DOCUMENTATION,  June,  1981." 

Selected  references  used  for  the  rewritten  version  of  DID-114 
are  as  follows: 

1 )  Atre ,  S .  Data  Base:  Structured  Techniques 
for  Design,  Performance,  and  Management. 

New  York:  John  Wiley  &  Sons,  1980. 

2)  Date,  C.J.  An  Introduction  to  Database  Systems, 

2nd  ed.  Reading,  Mass.:  Addison-Wesley,  1977. 

3)  Ullman,  Jeffrey  D.  Principles  of  Database 
systems.  Potomac,  Maryland:  Computer  Science 
Press,  Inc.  1980. 

General  Comments  on  R-DID-114  follow. 

1.  In  general,  the  DID  suffers  from  a  lack  of  focus,.  The  Data  Base 
Design  Document  identifies  itself  as  covering  both  the  data  base 
application  system  and  the  data  base  management  system  (DBMS). 
The  delineation  of  documentation  between  each  of  these  distinct 
products  should  be  explicit.  The  DID  allows  for  inclusion  of 
standard  documentation  on  the  DBMS  portion  whei  documenting  a 
data  base  application.  But  the  DID  confuses  this  by  selecting 
out  and  interleaving  parts  of  a  DBMS  description  with  the 
application  data  base  descriotion.  For  example  paragraph  3.3 
Data  Base  starts  with  the  requirement  for  describing  the 
terminology  and  relationship  between  the  different  groupings  of 
data.  These  are  elements  of  a  DBMS,  independent  of  the 
application.  It  continues  with  a  requirement  to  name  all  files, 
records,  fields,  and  items,  which  are  dependent  on  the  applica¬ 
tion.  It  then  follows  with  the  requirement  to  identify  con¬ 
struction  and  reconstruction  algorithms,  an i  data  access  and 
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documentation. 

2.  There  is  a  definite  need  to  make  the  documentation  required 
distinct  for  the  two  areas  of  data  base  applications  and  data 
base  management  systems.  This  provides  the  means  to  clearly 
identify  implementation  dependent  issues  distinct  from  applica¬ 
tion  requirements. 

3.  There  is  no  attempt  to  address  any  of  the  many  additional  issues 
in  data  base  design  resulting  from  implementation  as  a  distri¬ 
buted  application.  Such  issues  include  data  concurrency,  data 
distribution,  distributed  query  processing,  distributed  control, 
aria  access  and  security  in  a  distributed  application.  While  the 
user  of  a  specific  application  would  not  necessarily  know  the 
difference  between  a  centralized  or  a  distributed  data  base 
implementation,  the  designer  of  such  an  application  would. 

4.  The  meaning  of  section  3.?.  is  not  clear.  Does  it  refer  to  the 
programming  source  language?  If  so,  than  that  information  does 
belong  here,  but  in  a  programmer  reference  manual.  Does  It 
refer  to  some  data  base  descriptor  language?  If  so,  then  it 
should  be  clarified.  It  Is  also  not  clear  why  section  3.2.2  is 
included  as  a  subparagraph  of  3.2  and  why  it  is  titled  "Processor 
Design." 

5.  The  description  of  the  data  base  should  not  be  a  subsection  of 
the  section  describing  the  data  base  management  rules.  The  infor¬ 
mation  asked  for  under  section  3.,  3.1,  and  3.2,2  is  redundant 
with  information  called  for  in  section  3.4. 

6.  The  description  of  the  data  base  elements  should  follow  the 
structure  of  the  data  base  itself.  The  following  outline  for 
section  3.1  is  suggested: 

3.  Data  Base  Description 

3.1  Data  Description 

3.1.1  File  Design 

3.1.1. W  File  W 

3.1.1. W.X  Record  X  (description  of  a  record  of  file  W) 

3.1.1. W.X.Y  Field  Y  (field  Y  of  record  X) 

3.1 .1 . W.X.Y.Z.  Item  Z  (item  of  field  Y) 

(identical  structures  should  reference  previous 
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descriptions  rather  than  repeating  them) 

3.2  Management  system 

3.2.1  Data  Base  Construction 

3.2.2  Qata  Base  Reconstruction 

3.2.3  Accessing  and  Manipulation 


A  WORKING  PAPER  ON 


DATA  BASE  DOCUMENTATION 


by 
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The  attached  Data  Item  Description  was  written  In  response 
to- an  assignment  b y  the  Chairman  of  Panel  "A"  of  the  Software 
Vorkshop  held  at  Monterey,  California  22-25  June  1981  at  the 
I'aval  Post  Graduate  School* 

The  panel  is  specifically  addressing  Software  Documentation 
on  behalf  of  the  Joint  Logistic  Commanders'  Policy  Coordinating 
Group  on  Computer  Resource  l-anagement «  The  objective  is  to 
reduce  the  size  and  amount  of  0 paper  requirements”  inherent  in 
modern  software  development.  This  particular  assignment  Is  to 
review  the  Proposed  Data  Base  Description  (DID)  which  is  denoted 
as  R-DID-114. 

cc: :  ztmtary  op  apip»:.i4 

Data  bases  have  been  a  most  neglected  and  misunderstood  part 
of  software  development.  It  is  seldom  recognized  that  a  data¬ 
base  may  be  many  magnitudes  more  conple;:  than  the  ‘"programs”  or 
algorithms  which’  drive  the  data  base. 

One  of  the  basic  problems  is  the  definition  of  just  what  a. 
data  base  is.  Host  definitions  suffer  from  the  "blind  man 
syndrome , "  as  described  in  the  fable  about  a  number  of  blind 
men  asked  to  describe  an  elephant.  Each  one  described  the  pare 
he  could  feel.  3o  it  is  with  data  bases.  The  data  base  require¬ 
ments  and  definitions  given  by  a  Management  Information  System 
Specialist  are  entirely-  different  from  those  given  by  an  appli- 
cations  programmer  who  works  only  with  tactical  embedded  computers. 

Another  complicating  factor  in  dealing  with  data  bases  is  the 
fact  that  originally  the  pioneer  designers  of  the  data  base  concept 
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tried  to  make  the  operating  environment  entirely  invisible  to  the 
user.  This  was  a  noble  ideal,  but  as  many  data  base  designers 
have  learned,  it  is  very  difficult  to  talk  about  a  specific  data 
base  without  considering  the  environment  it  is  being  used  in. 

Falling  to  do  this  has  caused  some  very  rude  subrises,  such  as 
the  illfated  hid  CCS  system.  hhliCCS  failed  because  its  data  base 
design  was  tied  to  a  hardware  suits  which  stood  still  in  time,  vkile 
the  user  requirements  grew,  and  grew',  and  grew.  The  sane  problem 
faces  any  da.ta  base  designer  today,  for  who  can  tell  what  surprise 
the  technological  explosion  will  give  us  next. 

he  have  learned  from  experience  that  no  natter  how  esoteric  or 
sophisticated  our  data  base  system  is,  it  ultimately  will  have  to 
go  on  a  suite  of  hardware  that  is  limited  in  size,  speed  and 
throughput  capacity,  thus  limiting  the  growth  potential  and  use- 
ability  of  its  data  base.  The  users  of  small  enbe-dded  computers 
have  had  to  relearn  what  the  users  of  big  systems  learned  a  decade 
ago,  the  fact  that  there  are  limits.  The  wise  designer  of  a  com¬ 
puter  data  base  today  scopes  his  design  with  the  thought  of  data 
conversion  constantly  in  the  back  of  his  mind. 

The  avowed  purpose  cf  the  current  workshop  is  to  reduce  the 
amount  of  paper  and,  thence , the  cost  of  computer  software  develop¬ 
ment.  This  must  not  be  acne,  hov/ever,  at  the  expense  of  losing 
the  far  more  valuable  asset  of  clear  and  maintainable  software 
documentation.  It  is  already  clear  that  the  cost  of  not  having 
this  asset  far  exceeds  the  cost  of  the  documentation.  Of  ffir 
greater  cost  saving  significance  is  the  establishment  of  a  clear 
and  commonly  understood  documentation  standard  which  describes  for 
all  users  what  the  documentation  requirements  are  and  allows  him/her 
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to  select  the  subset  vhich  best  suits  the  particular  application. 

The  DoD  goal  of  life  cycle  development  cost  reduction  can  best  be 
net  by  developing  a  good  Data  Base  Documentation  Standard  which 
is  clear,  flexible  enough  and  adequate  to  meet  both  current  and 
forseeable  needs  and,  thus,  obviate  the  necessity  for  creating 
unique  DIDs  or  by  confusing  the  industry,  DoD  and  the  user  by 
having  a  large, confusing  and  diverse  set  of  "specialized"  and/or 
general  "standards 

This  writer  has  reviewed  both  the  current^ proposed  and''. the 

past  DIDs  which  have  been  written  on  the  subject  of  Data  Base 

Design  and  does  not  find  o.r.y  of  these  adequate  to  fulfill  the 

re  “air event s  that  are  described  in  this  paper.  A  number  of 

tents  have  also  been  reviewed  on  this  subject,  the  last  of  vhich 

is  included  in  a  short  bibliography  attached  to  this  paper. 

present  concepts 

This  research  lias  revealed  a  thread  of  commonality  which  ties  / 
together  in  principle,  but  this  thread  is  seldom  recognized. 

The  result  of  these  endeavors  is  attached  to  this  paper. 

It  is  a  rewritten  version  of  R-DXD~ll4  in  the  format  which  this 
writer  believes  best  addresses  the  real  Issues  of  data  base  design 
and  management.  The  writer  does  not  entertain  ir.  the  slightest  way 
the  concept  that  this  document  is  anything  more  than  a  very  rough 
draft  or  that  it  is  sufficient  in  itself.  The  writer  does  feel, 
however,  that  it  does  address,  on  a  bread  spectrum,  the  real 
issues  of  data  base  design  in  a.  manner  that  would  adequately  meet 
the  current  and  at  least  some  of  the  future  requirements,  while  at 
the  same  time  deleting  many  of  the  older  architectural  "provin¬ 
cialism*?  of  the  preceding  decades.  The  draft  Is  intended  to 
enlighten,  stimulate  the  thinking  processes, and  to  invite  commentary. 
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Data  Base  Design  Document 
2.  Description/Purpose 

'  The  Data  Base  Design  Document  establishes  and  defines  the 
type  of  data  base,  the  environment  in  which  it  is  used,  its 
design/architecture,  its  structure  and  the  methods,  rules  and 
language  required  for  the  maintenance,  update  and  use  of  the 
data  base  system. 

See  Section  (7)  on  separate  sheet  ^.la) 

1C.  Preparation  instructions 
10.0  Legal  preface,  see  DI-R-2174 

10.1  Identification  (See  R-DID-114) 

1C. 2  Introduction 

10.2.1  Background  Describe  the  type  of  system  this  particular 
data  case  is  intended  to  be  used  on  ana  the  use/user  environment. 
Clear  distinction  shall  be  made  as  to  whether  the  data  base  is 
intended  for  use  as  a  man: ge. went  information  system,  a  scientific 
application,  an  embedded  computer  system,  or  other  system  of  special 
design  and  purpose. 

10.2*2  Purpose  Describe  the  purpose  and  intent  of  “.he  Data  Ease 
Document.  formally  this  should  be  the  same  as  block  3  above,  but 
tailored  and  enhanced  to  fit  the  particular  application  of  this 
document. 

10.2.3  Scone  This  paragraph  shall  describe  the  scope  of  the 
data  base  and  its  objectives  in  terms  of  theuusc-r  requirements, 
mission  requirements  or  system  requirements  and  the  manner  in 
which  these  objectives  are  intended  to  be  met  and  described  by 
this  document. 

10.3  Type  o£  Data  Base  This  section  shall  describe  in  detail 
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the  type  of  data  base  that  is  intended.  This  description  shall 


use  the  terminology  and  vernacular  appropriate  to  the  application 
of  .the  system  described,  with  the  intent  that  it  be  clearly 
understandable  by  the  practitioners  of  the  discipline  addressed. 

10.3.1  Hardv: are  Environment  This  section  shall  describe  the 
basic  hardv/are  environment  v;hich  the  data  base  system  is  intended 
to  be  used  on.  It  v/ill  describe  the  type  of  computer,  the  memory 
allocation  for  data,  the  I/O  ports  available,  Data  Base  System, 
and  types  of  peripherals  intended.  Where  distributive  processing 
is  used,  the  number  of  processors,  the  netyjork  interconnect  scheme, 
the"hardshake  protocol"  and  specific  constraints  shall  be  addressed. 
When  appropriate  and  convenient,  these  documents  shall  reference  the 
resource  project  on  other  documentation  which  contains  the  cited 
information. 

10.3.2  Softy.’? re  Environment  This  section  shall  describe  the 
softy/are  environment  y.’hich  the  data  base  system  y/as  designed  to 
be  used  on.  If  the  system  is  designed  for  general  use,  it  shall 

so  state,  but  shall  specify  the  systems  oh  which  it  has  been  tested 
and  used  and  describe  constraints,  if  any  which  are  peculiar  to 
each  system. 

10.3.2.1  Operating  System.  Where  appropriate,  name  and  describe 
the  operating  system  which  the  data  base  was  designed  for  use  with. 
This  description  should  describe  the  setup  procedures,  job  control 
language  required,  utility  programs  required  and  or  other  special 
configurations  used^such  as  special  library  packages  and  the 
special  user  software  required.  This  section  shall  state  the 
High  Order  Language  (HOL)  or  other  which  the  user  program  is 
written  in  and  the  version/nod  of  the  computer  HOL  required  to 
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compile  these  programs.  V/here  the  system  is  designed  for  use 
under  5  special  run  time  executive  written  far  an  embedded  computer, 
user  constraints  shall  be  specified 

10.3.3  Data  Base  Design  Tyne  This  paragraph  shall  address  the 
particular  design  approach  of  the  data  base.  In  particular,  it 
shall  state  the  design  approach  used,  whether  the  relational 
approach,  the  hierarchial  approach,  or  the  network  approach  or  what 
other  approach  is  being  utilized. 

10.3.3.1  Record  Characteristics  Describe  the  record  and  file 
characteristics  of  the  data  base  design.  The  definition  of  such 
words  as  file,  record,  etc.,  shall  be  given.  Variable  or  fixed 
length  file/record  schema  shall  be  described.  Also  describe  the 
basic  record  characteristics  and  type,  such  as  byte,  word,  etc.. 

In  general,  the  purpose  and  intent  of  this  section  is  to  fully 
describe  all  record  data  characteristics.  The  description  shall 
include,  but  not  be  limited  to,  the  following. 

10.3.3.2  Record  Access  I.'ethods  Describe  the  type  of  record  access 
methodology  used,  such  as  sequential  or  indexed.  Describe  the 

data  reference  dictionaries,  data  indices,  sequence  pointers, 

dictionaries,  and  ,  .  .  , 

data^bibliographies.  In  particular,  show  a  record  map  v/hich 

describes  where  pointers  are  embedded  into  the  basic  record  unit. 

10.3.3.3  Data  compression  and  Expression  If  data  compression 
and  expansion  techniques  are  used,  describe  the  methodologies  and 
techniques  used  to  compress  and  decompress  the  data. 

10.3.3.4  Stacking  hliere  stacking  and  stack  processing  4re  used, 
describe  fully  the  stacking  techniques  used,  and  the  stack  table/ 
pointer  structure. 
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10.4.1  Throur.hnut/t lining  requirements  This  paragjyjhh  shall 

specify  the  throughput  requirements  and  constraints  in  terms  of 

number  of  record  sets  per  unit  time.  Where  applied  to  an  embedded 

computer,  this  section  shall  specifically  state  the  access  time  cf 

a  unit  record,  as  specified  by  the  contract,  from  mess  stroage, 

through  buffer  channels,  and  to  the  active  memory  use  via  the 

Tontine  executive  or  direct  memory  access  used.  This  unit  time 

shall  then  he  used  to  shov  full  access  time  in  terms  of  system 

time  available  and  number  of  records  that  can  be  processed  in 

such  a  manner  that  total throughput  times  can  be  computed.  The 

specific  data  base  access  time  and  reserves  time  shall  be  stated. 

Where  random  record  access  techniques  are  used  and  where  applicable, 

this  shall  be  stated  in  terms  of  a  statistical  equation  which 

addresses  systems  design  parameters,  in  such  a  way  as  to  allow  a 

determination  cf  the  effect  of  system  expansion  on  timing  and 

throughput.  Where  distributive  systems  are  used  the  impact  of 
for 

schEoe/br  lor  it  iz  ing  of  data  access,  and  the  "handshake  protocol"  used 
shall  be  considered  with  regard  to  their  impact  arritiming  and 
throughput.  Where  the  application  addresses  a  management 
information  system,  this  paragraph  shall  show  the  anticipated 
throughput  with  respect  to  the  target  user  system.  It  is  the 
intent  of  this  paragraph  to  enable  the  user  to  assess  the  operational 
and  cost  impact  of  increasing  data  base  sice* 

10.4.2  User  Format ing  Requirements  Where  the  data  base  described 
is  for  management  information  systems,  or  business  type  application, 
this  section  will  be  included  to  describe  for  the  user  the  required 
user  information. 

10,4.2.1  f WpM  nn  OvpttH  pw  Show  an  overall  flow  diagram. 
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Describe  the  hierarchy  of  data  base  management  rules  and  how 
they  are  implemented. 

10.4,2*2  User  Language  Describe  the  syntax  and  rules  for  the 
use  of  the  full  management  language.  A  complete  dictionary  of  user 
words  and  meanings  shall  be  included.  If  a  particular  syntax, 
ouch  asthe  Bacus  Naur  (BKF)  syntax  is  used,  it  shallbe  completely 
described,  or  an  appropriate  reference  document/c  it  eel  vlich 
will  acceomplish  this  purpose.  In  any  case,  departures  from 
standard  syntax  shall  be  fully  explained. 

10.4.2.3  Out cut  Formats  The  format  of  all  output  data  shill 
be  described  in  terms  of  information  fields  and  the  purpose, 
meaning  and  intent  of  the  information  presented. 

10.4.2.3  Data  Incut  Formats  This  section  shall  describe  the 
input  data  format  in  terms  of  the  outputs  they  generate,  \\here 
the  output  format  is  variable,  the  controls  and  tags  for  varying 
the  output  and  the  expected  result  shall  be  described. 

10.4.2.4  Data  Addess  Where  applicable  and  in  the  use  of  a  specific 
environment,  the  methodology  of  calling  and  displaying  the  data 

via  an  interactive  terminal  shall  be  described. 

10.4.2.4.1  Special  Security  Describe  all  data  security 
requirements . 

10.4.2.5  Storage  Requirements  This  section  shall  describe  all 
data  storage  requirements  which  shall  include  active  memory, 
peripheral  storage  requirements,  and  backup  storage  requirements. 
Thos  requirements  shall  be  described  in  terms  of  the  storage 
required  and  or  used.  Backup  storage  life  expectancy  and/or 


renewal  cycles  shall  be  recommended^  and  the  arguments  supporting 
these  requirements  presented. 

10.4.2.6  Human  Tnt.prf.q_re  Regnirenants  ihis  section  shall  describe 
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the  method  of  collecting  data  for  organ i ::ir.g,/ date,  collected  £r^  the 
method  of 

/  input  ir.g  to  and  updating  the  data  base, 

10,4,2,6,1  human  Factors  This  paragraph  shall  address  the 
complexity  level,  human  comprehensibility,  and  training  reouired 
for  personnel  support  and  use  of  the  data  base, 

10,4,7  Oats,  Conversion  dec u  Ire  merits  This  section  shall 

address  specifically  the  methods  and  requirements  for  converting 
the  data  base  or  specific  components  cf  the  data  base  to  another 
data  b:se.  This  section  shalf describe  the  characteristics  of  the 

addressed  data  base  such  as  /.IT3CI  cod;!,  Field  data,  F2CLIC ,  etc. 

ar d  thc  re",  uirc'snts  for  convertin'^ 

/from  one* type" o?~ date” base  to" another ,  It  shall  include  changes 

in  media  ta~s,  identification  ana  formating  labels,  such  as 

tppe  leader  formats,  end  of  file  morns,  etc..  This  section  is 

mandatory  and  applies  uo  ;he  format  of  thc  total  data  delivered 

in  the  final  program  product  pac I:ag<;. 

10,o  u"eci-'i  fat-  i-,c<  ulrccnts  iihan  called  for  in  the  contract 
this  section  shall  adaresa  r.y  sped.al  data  requirements  not 
covered  in  the  preceding  sections.  It  may  also  be  used  at  the 
■discretion  of  the  contractor  as  specified  in  the  contract  to 


describe  the  internal  active  data  memory  structures  and/or 
forms ts  of  internal  data,  The  general  format  of  the  data  shall 
include  but  not  be  restricted  to  ohe  following  sections, 

10.5.1  Data  Diet ionar.v  This  shall  give  the  definition  ana 
description  ofthe  internal  use  o i:  all  internal  data.  Data  shall 
be  grouped* and  defined  as  the  following,  as  a  minimum: 

(a)  Global  Data:  Defined  as  data  used  by  a  number  of 
processors  operating  as  e  system; 

(b)  Common  Data:  Defined  as  data  used  connonly  by  a 
number  of  modules  viuhin  one  processor; 
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Cc)  Local  Data:  Local  data  is  data  used  by  a  single 
nodule  or  routine. 

„  All  data  shall  be  listed  by  its  liberal  name  and  its  symbolic 
name  and/ar  the  engineering  mathematical  symbol  it  represents.. 

Data  shall  further  be  listed  in  accordance  with  its  functional 
characteristics,  shch  as  whether  It  is  ASCI  data,  Boolean  logical 
data,  or  mathematical  data  formated  for  external  transfer  or  Internal 
usage  by  the  CPU.  If  the  data  is  arithmetic  in  nature  the 
scale  factor  shall  be  specified  and  the  computer  word  size 
requirements  stated,  along  with  field  definitions  within  the 
data  v;ord,.  such  as  the  mantissa  and  characteristic  of  an 
arithmetic  double  precision  float ing  point  word. 

In  addition,  data  shall  be  categorized  as  to  whether  it  is 
constant  data  or  variable  data,  and  all  v-riable  data  shall 
rpecifically  st§te  wherfrier  or  not  an  initial  value  is  required 
and  what  that  value  should  be  along  with  any  special  conditions 
which  req  tire  its  reinitialization. 

10. 5.2  Data  Cross  Reference  Where  required  a  Data  Cross 
Reference  shall  be  provided. 

10.5.3  Memory  Kan  A  memory  map  shall  be  provided  which  shows 
the  location  in  memory  where  all.  data  are  stored. 

10. 5*^  Table  Structure  This  section  shall  describe  the 
structure  of  all  internal  tables  (meaning  tables  internal  to 
a  specific  CPU' memory  structure).  The  structure  of  the  table 
shall  describe  the  table  access  dictionary,  the  basic  memory 
storage  unit  (i.e.,  1  byte,  2  byte,  etc.).  The  indexing 
scheme,  the  table  access  dictionary,  and  table  concatenation 
schemes.  V/here  practical  these  shall  be  standardized  throughout 
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the  system. 

10,?. 5  Special  ’.'or:.  Formats  This  section  shell  describe  any  or 
all  special  word  formats  used  in  the  data  base.  For  example, 
see  Figure  I  for  an  example  of  a  data  word  used  to  depict  a 
rqnge  and  elevation  word  used  in  a  tactical  computing  system. 
(Use  Tigure  I  in  DI-A-2140) 

Other  examples  nay  be  packed  word  instruction  formats  used  in 
"table  driven"'  control  systems. 
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(7)  Application/ Interrelationships  This  document  Is  based  on  the 
top  level  computer  program  specification  documents  and  is 
intended  to  be  used  with  large  software  development  projects 
and  with  a  variety  of  software  applications.  It  is  Intended 
that  it  be  tailored  to  fit  the  particular  applications  and  is 
written  to  cover  a  variety  of  disciplines,  from  the  large  data 
bases  to  small  embedded  computer  systems  or  distributive  com¬ 
puter  systems. 

The  elements  of  the  DID  are  intended  as  a  minimum  set  of 
data  which  the  user  may  tailor  to  his/her  specific  requirements. 

Block  (9)  MIL-STD-483 
MIL-STD— 1 679 
DoD  Standard  7935.1-5 


F~40 


R-OID-1 15/SOFTWARE  PRODUCT  SPEC  (SPS) 


1.  2.3 

line  3 


Add  after  disk,  “or  firmware  memory 
device". 


2.  3.2  This  section  calls  for  listings  if  IDDs  are 

generated.  Listings  are  not  associated  with 
IDDs.  The  IDD  only  specifies  an  interface. 
The  software  to  implement  the  Interface  is 
included  in  the  interfaced  CSCIs. 


The  Panel  was  equally  divided  on  the  following  comments. 

3.  The  "model"  upon  which  the  IDD  (and  the  IRS)  Is  based  is  needed 
by  a  reviewer  In  order  to  appreciate  what  Interface  listings 
are  (see  3.2  of  the  DID).  An  Interface  is  the  description  of 
the  functional  (data  transferred)  and  physical  (electrical 
signals  that  go  between)  characteristics.  Within  each  CSCI 
there  will  be  programs,  such  as  message  handlers,  that  handle 
the  data  transferred  from  another  CSCI.  The  listing  of  such 
programs  would  not  go  in  an  Interface  documen'  but  in  the 
product  spec  of  the  CSCI  In  which  the  program  is  contained. 

4.  Source  comment  requirements  should  be  put  in  the  system/segment 
spec  and  then  allocated  to  the  SRS  which  is  the  "test  to"  docu¬ 
ment.  That  way  the  listings  can  be  tested  (by  inspection)  to  see 
if  they  conform  to  the  good  workmanship  requirements  that  were 
specified.  This  is  no  different  than  the  workmanship  for  hard¬ 
ware  that  is  already  called  for  in  the  system  spec  (3.3.3,  3.3.3. 
2,  and  3. 3. 3. 3). 

5.  2.3  Delete  from. . "Compiler  Source  statements..." 

to  end  of  page. 

6.  Page  2  Delete  from  top  of  page  thru  "The  Restrictions" 

sentence. 


7.  3.2  IDD  should,  if  anything,  be  appended  to  IRS. 

8.  2.3  Cross  reference  listings  specifically  call  for 

mnemonically  labeled  statements.  Numbered 
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statements  should  also  be  cross  referenced. 
Change  the  bullets  to  sub-paragraph  numbers. 

3.  A  provision  needs  to  be  made  for  machine  readable  object  code. 

In  section  2.3,  delivery  on  "card  decks,  magnetic  tape,  or  disk" 
is  too  restrictive.  What  about  direct  transmission  across  remote 
data  lines,  or  pluggable  bubble  memories,  etc.?  Perhaps  a  phrase 
like  "media  acceptable  to  the  Government"  could  be  used. 
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.1  i.tiAiA*:, 


R-DID-116  VERSION  DESCRIPTION  DOCUMENT 


This  HID  is  acceptable  as  is,  although  some  panel  members  are  in 
favor  of  the  following  change. 

Paragraph  2,  change  "APPLICABLE  DOCUMENTS"  to  "REFERENCE  DOCUMENTS" 
since  a  Version  Description  Document  does  not  have  "applicable 
documents." 
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R-DID-117  SOFTWARE  TEST  PLAN 


1.  A  preliminary  test  plan  is  needed  at  POR  and  the  final  plan 
should  be  submitted  at  CDR.  (Reference  80.6451.2-316,  Rev.  1, 
page  27). 

2.  Block  7  line  4  Change  "Quality  Assurance"  to 

"Test". 


3.  Block  7  line  6 


4.  4.1 


5.  4.2.2  (a)  line  3 

6.  4.2.4  (a) 

5.2.1  (a) 


Change  period  following  "Specifica¬ 
tion"  to  comma  and  make  "Also" 
lower  case. 

This  isn't  the  place  to  define 
a  unit.  It  should  be  defined  in 
the  SOP  or  SSPM. 

Add  "testing"  after  "integration". 
Define  terms. 


7.  4.2.5. b  This  section  should  reference  the 

SRS  as  the  source  of  the  software 
reqiri  remen  ts . 

8.  5.2.3  The  example  of  a  tabular  form  to 

correlate  tests  versus  requirements 
could  be  better. 


9.  5.5.d 


Shouldn’t  the  "of  control"  be  "to 
control"? 


SPLIT  DECISION 


The  panel  was  equally  divided  on  the  following  comments. 

10.  Paragraph  3  Line  5  Software  Test  Plan  es  not  account 

for  system  level  testing  and  it 
should. 
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11.  The  test  plan  should  provide  for  both  informal  and  formal  Hard¬ 
ware/Software  Integration  Testing.  These  tests  should  be  design¬ 
ed  to  verify  that  the  software  correctly  functions  with  the  inter¬ 
facing  hardware  prior  to  system  level  test.  In  many  cases 
system  level  tests  do  not  adequately  test  the  software/ hardware 
interface  from  the  software  viewpoint. 

12.  This  DID  refers  to  interfaces  between  the  contractor  and  the 
government  (seems  to  use  customer  and  government  interchangeably). 
It  should  also  specify  an  interface  to  contractor  CM,  QA,  and 
Independent  test  groups.  There  is  also  an  Independent  Verifica¬ 
tion  and  Validation  contractor  interface  when  used. 

13.  There  should  be  one  outline  for  all  three  types  of  testing  (Unit, 
Integration  and  Qualification  testing)  a  proposed  outline  would 
be: 

x.1  Purpose 
x.2  Resources  Required 
x.3  Test  Management 
x.4  Test  Structure 
x.5  Test  Requirements 
x.6  Schedules 
x.7  Retest 
x.9  Test  execution  QC 

14.  Block  3  Line  4 

15.  1.1  Line  4 

16.  Paragraph  2 

17.  Paragraph  3  Line  5 


"test  requirements"  are  provided 
In  SRS. 

"Authority"  not  called  for  In  other 
DIDs. 

Some  provisions  should  be  made  for 
reference  documents. 

The  word  "procedures"  is  used  as 
part  of  the  description  or  purpose 
of  this  document  but  it  is  not. 

Test  Procedures  in  a  separate  DID. 
Remove  the  phase  "test  implementa¬ 
tion  procedures". 
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10.  Paragraph  4 


If  contractor  testing  is  informal,  i.e., 
not  contractually  deliverable,  why  is  it 
so  explicitly  defined  in  a  contractual 
document. 

Move  to  Standards  and  Proceaures  Manual . 


19.  4.1  a  thru  d 


R-DID-118  SOFTWARE  TEST  PROCEDURES 


1.  General  Comment:  An  apparent  conflict  exists  between  this  DID 
and  R-DID-119,  Test  Report.  119  is  geared  to  an  integrated 
software  system  while  DID  118  addresses  module  or  task  level 
type  testing.  DID  119  states  that  the  test  procedures  applicable 
shall  be  part  of  test  report.  Is  one  test  procedure  meant  to 
accommodate  all  levels  of  testing  or  are  they  oriented  to  a  test 
level . 

2.  Paragraph  7  Line  5  Change:  "base"  to  "basis" 

3.  Paragraph  2  Change  Title  from  "Appropriate  docu¬ 

ments"  to  "Reference  Documents." 

4.  Paragraph  10  Line  7  Change  "a  test  procedure"  to  "test 

procedures." 

5.  Paragraph  1.1  Line  4  Change:  "base"  to  "basis" 

6.  Paragraph  2.0  Line  2  Change:  "base"  to  "basis" 

7.  Paragraph  9.2  Change  title  to  "Computer  Preparation" 

because  "Digital"’  excludes  Hybrid  or 

Analog  Computers. 

8.  Paragraph  10.0  Change  "a  detailed  procedure"  to 

"detailed  procedures." 
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R-DID-119  SOFTWARE  TEST  REPORT 


1.  Paragraph  1.1  Line  3  Change:  "test  procedure  and" 

"test  plan,  test  procedure, 
and"  to  complete  the  defini¬ 
tion  of  the  test  being 
reported 

2.  Paragraph  1.1  Line  4  Change:  "Also  identify"  to 

"Identify"  for  clarity  and 
to  prepare  for  following 
commands 

3.  Paragraph  1.1  Line  5  Change:  "participate"  to 

"participated" 

4.  Paragraph  1.1  Add  after  last  sentence:  "Identify 

date  and  location  of  test." 

5.  Paragraph  7.0  Line  2  Change:  "design  or  operation"  to 

"design,  operation,  or 
additional  testing"  to 
provide  capability  to 
recommend  the  conduction  of 
additional  test  to  fulfill 
objectives  for  which  results 
were  not  expected. 


R-DID-120  COMPUTER  SYSTEM  OPERATOR'S  MANUAL 


1.  Block  7  and  10  (1.1) 

2.  Block  9 


3.  Block  10  Line  2 

4.  1.2a 

5.  3.1 .b.3 

6.  4.1  Line  2 

7.  General 


Delete  "host" 

Reference  should  be  made  to  MIL-STD 
-XXX  being  developed  by  the  JLC's 
under  contract  that  will  supersede 
MIL-STD-1679.  MIL-STD-1679  is  a 
Navy  document  and  not  recognized 
by  all  services  under  the  JLCs. 

Add  "and"  at  end  of  line. 

Change  "by  name  and  nomenclature" 
to  "either  by  popular  trade  name 
or  official  nomenclature." 

Lowercase  bootstrap 

Delete  . .if  any. . ." 

The  manual  should  be  verified. 
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DID  121  -  SOFTWARE  USERS  MANUAL 


1.  This  DID  is  written  for  Air-Force  use  and  not  joint  service 
JLC  use.  It  specifically  references  AFSCM/AFLCM  375-7  where 
it  should  reference  MIL-STD-XXX  being  developed  by  the  JLC. 

2.  Paragraph  5,  obviously  written  with  a  batch/operat'ing  system  in 
mind.  Should  be  r:.ide  flexible  enough  to  cover  tactical  software 
and  console  operations  of  a  complex  nature. 

3.  Block  3,  correct  spelling  software. 

4.  Correct  punctuation  throughout. 

5.  Paragraph  4. a, 7.,  delete  "formalized". 

6.  Add  a  paragraph  tc  specify  system  termination  procedures. 
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DID  122  -  COMPUTER  SYSTEM  DIAGNOSTIC  MANUAL 


1.  The  manual  should  also  Include  a  fault  catalog  listing  error 
messages  that  are  presented  to  the  operator  by  built  in  test 
software.  Such  a  catalog  should  include  instructions  for  manual 
test  using  special  equipment  or  replacement  card  numbers. 

2.  Block  3  of  the  DID  says  that  the  Diagnostic  Manual  is  to  identify 
hardware  malfunction.  This  Is  consistent  with  the  common 
meaning  of  a  diagnostic  Manual.  However  Block  7  says  that  the 
manual  Is  based  on  the  SRS.  It  should  be  based  on  the  Software 
Programmer's  Manual  (R-IJID-123) . 

3.  Block  7  states  that  a  Software  User's  Manual  is  not  required  when 
a  diagnostic  manual  is  written.  These  documents  are  not  related; 
one  shouldn't  influence  the  other.  Users  and  maintenance 
personnel  are  usually  (Afferent  people  with  different  responsi¬ 
bilities  and  background. 

4.  Paragraph  3.  Circled  iame/Nomenclature.  Question  What' s  the 
difference  -  redundant,. 

Minority  Op inion 

There  is  no  need  for  a  separate  DID  for  a  Diagnostic  Manual.  Applying 
the  Software  User's  Manuel  DID  to  the  diagnostic  CSCI  will  achieve  the 
same  thing. 
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DID  123  -  SOFTWARE  PROGRAMMERS  MANUAL 

1.  Block  7  -  meaning  of  "Interpretive  Computer  Simulation  (ICS)"  is 
questioned. 

2.  Block  9.  This  DID  is  written  for  AF  use  and  not  joint  service 
JLC  use.  It  specifically  references  AFSCM/AFLCM  375-7  where  it 
should  reference  MIL-STD-XXX  being  developed  by  JLC. 

3.  Paragraph  1.0  -  The  scope  of  subject  manual  is  specified  in  para¬ 
graphs  1.1  and  1.2  conflicts  with  text  contained  in  Blocks  3  and 
7  which  appear  to  require  a  computer-dependent  manual  vice  a 
system-dependent  manual.  Since  a  given  computer  may  be  imple¬ 
mented  in  many  systems  it  is  recommended  that  subject  manual  be 
system-independent.  This  will  tend  to  preclude  the  time-honored 
technique  of  re-documenting  the  system-independent  material  with¬ 
in  system  dependent  documentation.  Suggest  documentation  be 
structured  to  prohibit  the  redocumentation  practice.  This  will 
permit  single-source  documentation  with  a  resultant  capability 
of  effective  configuration  control. 

4.  Paragraph  5.2.C.10  -  Inflating  terminology  should  be  defined  if 
used  (e.g.,  "supercode"). 

MINORITY  REPORT 

5.  The  DID  for  the  Software  Programmers'  Manual  is  not  very 
useful  as  it  is  written,  because  it  is  only  applicable  to  the 
macro  programming  level.  This  type  of  manual  will  be  less  and 
less  needed  with  the  increase  in  use  of  standard  architectures 
and  higher  order  languages.  There  is  a  definite  need  for  a  pro¬ 
gramming  manual  which  is  oriented  from  the  microcode  level,  down 
to  algorithms  implemented  in  discrete  logic  and  driven  by  PROM 
date.  (AYK-14  is  currently  dealing  with  the  former  case,  and 
LAMPS  is  dealing  with  the  latter  case.)  Attached  is  a  proposed 
Microcode  Programmers  Reference  Manual  DID  written  for  the  AYK-14. 
Whether  it  is  better  to  combine  this  with  the  existing  SPM  DID  or 
leave  them  separate,  I  do  not  know. 


F-52 


filoPoseb 

DATA  ITEM  DESCRIPTION 

1.  Title: 

Microcode  Language  Reference  Manual 


3.  Description/Purpose 

The  Microcode  Language  Reference  Manual  (MLRM)  provides  instructions  to  enable 
microcode  programmers  to  prepare,  interpret,  and  alter  microcode  programs. 
These  microcode  programs  will  be  written  in  a  particular  machine,  assembly  or 
compiler  language,  usually  for  a  specific  computer.  The  manual  is  based,  in 
part,  on  the  product  performance  and  design  specifications  which  describe  the 
particular  machine. 

7.  Application/Interrelationship 

This  Data  Item  Description  is  used  to  develop  a  microcode  language  reference 
manual  for  a  specific  microcode  language  being  developed  where  no  suitable 
commercial  manual  exists.  Microcode  is  machine  code  which  controls  the 
elementary  parts  of  the  computer  to  define  the  instruction  set  architecture. 
The  MLRM  will  be  used  for  evaluating,  generating,  correcting,  or  updating  one 
or  more  microcode  programs.  This  manual  together  with  associated  microcode 
documentation  should  provide  a  sufficient  basis  for  microcode  maintenance.  It 
is  also  used  to  evaluate  commercial  manuals  to  ensure  they  meet  Navy 
requi rements . 

This  Data  Item  may  be  used  in  conjunction  with  DIDs: 

DI-S-2141  Program  Package 
Ol-E-2136  Program  Design  Specification 
DI-E-2138  Program  Performance  Specification 
DI-S-2139  Program  Description  Document 


10.  Preparation  Instructions 


The  MLRM  shall  contain  sufficient  information  to  enable  the  microcode 
programmer  to  fully  understand  the  microprogramming  aspect  of  the  particular 
machine,  assembler  or  compiler  language.  The  manual  shall  be  applicable  to 
any  microcode  program  and  any  equipment  configuration  of  the  computer  in  which 
that  microcode  program  may  be  utilized.  Information  presented  in  the 
MLRM  shall  be  that  necessary  to  enable  the  programmer  to  (1)  produce  a 
microcode  source  program  suitable  for  the  particular  machine  language  or 
assembler/compiler,  ;(2)  interpret  an  existing  microcode  program  for  that 
machine  language  or  assembl er/compi 1 ier ,  and  (3)  effect  corrections, 
additions,  and  deletions  in  microcode  programs.  Information  shall  be  in  the 
form  of  text,  supporting  illustrations,  tables,  diagrams,  and  appendixes. 

1.  Arrangement.  The  microcode  language  referenc  manual  shall  contain  the 
following  in  the  order  indicated: 

Title  Page 

List  of  Effective  Pages 
Table  of  Contents 
list  of  Illustrations 
Li st  of  Tab! es 
Introduction 

Chapter  1,  General  Information 

Chapter  2,  Programming  Features 

Chapter  3,  Bus  Control  and  Internal  Memory  Usage 

Chapter  4,  Program  Instruction  Statements 

Chapter  5,  Micro-instruction  Timing 

Chapter  6,  FPLA/PROM  Code  Generation 

Chapter  7,  Error  Detection/Oiagnostic  Features 

Glossary 

Appendixes 

Index 


2.  Introduction.  The  introduction  shall  identify  the  specific  microcode 
language  by  Government  or  manufacturer's  designation  and  shall  contain  a  brief 
statement  of  the  language's  use.  The  scope  of  the  manual  shall  be  outlined 
briefly  by  description  of  the  contents  of  each  chapter  so  that  the  user  can 
readily  locate  needed  information.  In  addition,  the  introduction  shall  list 
the  publications,  Government  specifications,  etc.,  needed  for  clarification  of 
symbols,  designations,  and  abbreviations  used  in  preparation  of  the  manuals. 

3.  Chapter  1,  General  Information: 

a.  Purpose.  This  portion  of  chapter  1  shall  contain  a  description  of  the 
purpose  of  the  manual  and  the  manner  of  its  intended  use. 

b.  Equipment  Organization.  This  portion  of  chapter  1  shall  consist  of  a 
brief  description  of  the  specific  target  computer  system,  including  a  general 
description  of  the  basic  modules  comprising  the  equipment.  The  information 
may  be  given  in  tabular  form.  A  figure  shall  be  provided  to  illustrate  the 
functional  equipment  organization.  A  block  diagram  of  the  computer  system 
shall  also  be  included  in  chapter  1.  Examples  of  basic  modules  are  the 
following: 

(1)  ALU 

(2)  Micromemory 

(3)  Internal  memory  and  FPLAs  other  than  micromemory 

(4)  Registers  used  by  microcode  other  than  the  internal  ALU  registers 

(5)  Busses  used  or  controlled  by  the  microcode 

(6)  Micro  sequencer 

(7)  External  interfaces  such  as  UARTS 

c.  Operational  Structure.  This  portion  of  chapter  1  shall  consist  of  a 
description  of  the  operating  characteristics,  capabilities  and  limitations  of 
the  target  computer  system  as  it  relates  to  the  programing  function.  The 
information  should  include: 

(1)  Machine  Cycle  Time 

(2)  Word  Length 
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(3)  Memory  Capacity  and  Characteristics 

(4)  Arithmetic  Operations 

(5)  Interrupt/Event  Capabilities 

(6)  Micro  Command  Word  Format 

(7)  Micro  Command  Word  Bit  Pattern  Definitions 

(8)  Module  Intercommunication 

(9)  Operational  Registers 

(10)  Internal  Flags  (parity,  overflow,  timeout,  etc.),  Flag  Registers, 
and  Control  Registers 

(11)  Input-Output  Descriptions 

(12)  Communication  With  Support  Equipment 

4.  Chapter  2,  Programming  Features: 

va.  This  ohaptar  snail  contain  descriptions  of  the  programming  features  of 
the  particular  microcode  language.  Complete  detail  should  be  included  if  the 
feature  is  unique  to  the  equipment,  machine  language  or  assernbl er/compi  1  er 
involved  and  could  not  be  expected  t,o  be  within  the  scope  of  knowledge  of  the 
experienced  programmer.  Features  described  should  include: 

(1)  Type  of  insi-ructions/statements 

(2)  Word  structure 

(3)  Command  list 

(4)  Pseudo-instructions 

(5)  Operand  stack 

(6)  Indexing 

(7)  Relative  direct  addressing 

(8)  Indirect  addressing 

(9)  Branching 

(10)  Subroutine  control 

(11)  Interrupt/Events 


b.  This  section  shall  include  illustrations  of  typical  instruction  word 
formats  and  data  word  formats  and  flow  charts  of  unusual  programming  features, 
as  applicable. 
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5.  Chapter  3,  Bus  Control  and  Internal  Memory  Usage 

a.  Section  1,  Bus  Control.  This  section  shall  describe  the  way  In  which 
the  microcode  controls  the  computer  busses.  Detailed  descriptions  of  the  use 
of  control  registers  and  flags  (set  or  cleared)  shall  be  provided.  These 
descriptions  shall  be  supported  by  Illustrations  and  Tables. 

b.  Section  II,  Internal  memory  Usage.  This  section  shall  list  all 
Internal  memory  locations  for  which  the  microcode  has  a  designated  use  and 
Indicate  their  use.  This  information  may  be  provided  In  the  form  of  a  Table. 

6.  Chapter  4,  Program  Instruction  Statements: 

a.  Section  I,  Microcode  Wora  Format.  This  section  shall  describe  in 
detail  the  word/ statement  formats  used.  An  illustration  of  each 
word/statement  format  shall  be  included. 

b.  Section  II,  Instruction  Description.  This  section  shall  describe  all 
the  micro-program  instructions/statements  used  with  the  computing  equipment. 
Examples  of  instruction  and  statement/group  are: 

(1)  Fixed-point  arithmetic  instructions. 

(2)  Floating  point  arithmetic. 

(3)  Shifting  operations. 

(4)  Hardware  Control  operations. 

(5)  Index  instructions. 

(6)  Logical  operations  (if  applicable). 

(7)  Input-output  operations. 

(8)  Branching  operations. 

The  execution  of  the  instructions  by  the  computer  shall  be  described  and 
illustrated  as  necessary,  with  flow  charts. 
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This  Information  shall  include 

(1)  Instruction  Mnemonic 

(2)  Description  of  What  The  Instruction  Does 

(3)  Micro  Command  bits  set  or  cleared  by  the  instruction 

(4)  Special  Conditions  When  This  Instruction  Can't  Be  Used 

Information  provided  shall  be  supported  by  tables  giving  descriptions, 
abbreviations,  and  codes  used  in  the  presentation.  Examples  illustrating  the 
use  of  the  Instruction  shall  also  be  provided.  This  section  shall  also 
include  a  table  of  instructions  with  variable  cycle  time,  a  table  of 
operational  registers  and  their  functions  and  table  of  arithmetic  properties 
of  registers.  ; 

7.  Chapter  5,  Micro-instruction  Timing.  This  chapter  shall  indicate  the  time 
it  takes  to  execute  a  complete  micro-instruction.  Any  special  conditions 
v/hich  cause  this  time  to  be  lengthened  or  shortened  will  also  be  stated.  This 
chapter  shall  also  explain  all  hardware  operations  not  completed  during  the 
same  micro-instruction  in  which  the  operation  was  initiated  (i.e.,  memory 
references  where  the  data  are  net  available  until  halfway  through  the  next 
micro-instruction) .  This  chapter  will  also  illustrate  the  timing  within  a 
micro-instruction  (i.e.,  if  the  micro- instruction  can  perform  more  than  one 
operation,  indicate  the  order  in  which  the  operations  are  performed). 

8.  Chapter  6,  FPLA  and  PROM  Code.  This  chapter  shall  contain  information 
Required  to  enable  tho  programmer  tu  write,  understand,  and  modify  the 
portions  of  microcode  wh’ch  generate  code  for  FPlAs  and  PROMs  other  than  the 
code  generated  for  micromemory.  This  shall  include  an  explanation  of  all 
instructions  used  that  are  not  defined  in  chapter  4,  illustrations  of  the 
format  required  for  these  instructions,  and  examples  which  will  illustrate  how 
to  generate  FPI.A/PR0M  code.  Diagrams  and  tables  shall  be  aaded  as  needed  for 
clari fi cation . 

9.  Chapter  /,  Listing  Interpretation.  Each  error  detection/diagnostic 
feature  of  the  compiler/ assembly  language  will  be  clearly  described.  The 
exact  printout  provided  by  the  system  will  be  shown  with  concise  descriptions 
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of  what  the  printout  means  and  what  conditions  generate  the  printout.  An 
explanation  of  any  cross  reference  list  and  memory  maps  generated  by  the 
ccmpiler/assembler  shall  be  provided. 

10.  Glossary.  The  glossary  shall  list,  in  alphabetical  order,  all  terms  and 
their  accepted  meaning  as  used  by  prograirmers,  operators,  and  other  personnel 
associated  with  the  target  computer  and  peripheral  equipment.  It  shall 
Include  definitions  of  registers,  components,  and  characteristics  of  the 
particular  equipment  which  the  manual  supports. 

11.  Appendices.  The  appendices  shall  contain  information  to  support  the 
programmer  In  the  preparation,  use,  and  maintenance  of  microcode  programs. 

Only  those  appendices  shall  be  included  which  are  required  for  the  particular 
computer  which  the  manual  supports.  Applicable  appendices  shall  be  numbered 
consecutively.  The  appendices  may  include: 

Appendix  1.  Mnemonic  listing  of  operation  codes  (if  applicable). 

Appendix  2.  Alphabetical  listing  of  operation  codes. 

Appendix  3.  Numerical  listing  of  operation  cooes. 

Appendix  4,  Instructions  by  operating  group  or  micro  coirmand  subfield. 
Appendix  5.  Listing  of  control  inputs  required  to  effectively  use  the 
machine  language  or  the  assembler/compiler. 

Appendix  6.  Magnetic  tape  BCD  codes  (if  applicable). 

Appendix  7.  Input-output  typewriter  codes  (if  applicable). 

Appendix  8.  Flexowriter  (or  equivalent)  codes  (if  applicable). 

Appendix  9.  Punched  card  codes  (if  applicable)  (should  include  an  example 
of  card) . 

Appendix  10.  Papertape  codes  (If  applicable)  (should  include  an  example  of 
tape) . 

12.  Index.  If  the  completed  manual  will  consist  of  more  than  50  pages,  an 
alphabetical  index  shall  be  provided. 
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13.  Arrangement  and  Content: 


a.  Abbreviations.  Standard  acronyms  and  abbreviations  may  be  used 
provided  they  are  first  defined  in  the  text.  They  shall  also  be  defined  in 
the  glossary. 

b.  Page  Numbers  and  Titles: 

(1)  Pages,  paragraphs,  figures,  and  tables  shall  be  numbered 
separately  and  consecutively  within  each  chapter  by  Arabic  numerals. 

(2)  Chapters,  numbered  paragraphs,  figures,  and  tables  shall  have 
brief  descriptive  titles.  Chapter  titles  shall  be  centered  horizontally  on 
the  page. 

c.  Space  Conservation.  Layout  shall  conserve  space  without  lessening 
usability  or  clarity  of  material.  Blank  pages  and  spaces  shall  be  avoided 
except  to  meet  basic  format  requirements.  For  example,  the  first  page  of  each 
chapter  must  always  start  on  a  right-hand  page,  even  though  this  may  require  a 
preceding  blank  page. 

d.  Blank  Page  Numbering.  All  blank  pages  shall  be  shown  on  the 
succeeding  printed  page,  under  the  number  of  that  page. 

I 

Example:  4-13 

(Page  4-12  blank) 

e.  Number  Identification  of  Manual.  Identifying  numbers  on  each  manual 
shall  be  as  specified  by  the  procuring  activity  and/or  user  activity. 

f.  list  of  Effective  Pages.  This  page  shall  provide  the  current  status 
of  each  page  of  the  manual  with  regard  to  its  letter  code  and  date  of 
publication.  If  a  manual  is  to  be  changed  periodically,  the  list  of  effective 
pages  shall  serve  as  a  control  device  by  specifically  identifying  the  latest 
changed  pages  as  well  as  showing  the  exact  page  breakdown  of  the  entire  manual. 
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g.  Table  of  Contents.  The  table  of  contents  shall  list  chapter  titles 
and  numbered  paragraph  headings.  When  applicable,  a  list  of  ill ustrations  and 
a  list  of  tables  shall  follow  the  table  of  contents. 

h.  Collating,  Drilling,  and  Binders.  Collating,  drilling,  and  the  type 
of  covers  shall  be  as  directed  by  the  procuring  activity  and/or  user  agency. 

1.  Style  of  Writing.  The  text  shall  be  written  In  a  style  that  Is 
clearly  understandable  to  the  users  of  the  manual.  The  Instructions  shall  be 
concise,  specific,  and  clearly  worded.  Illustrations  and  tabular  data 
(tables)  shall  be  used  as  necessary  to  clarify  or  supplement  the  written  text. 

j.  Authorization  for  Relaxed  Format  and  Reproduction  Requirements. 

Relaxed  format  and  reproduction  methods  shall  be  permitted  for  the  handbooks 
In  the  Interest  of  economy  and  expeditious  availability.  Areas  in  which 
requirements  may  be  relaxed  are: 

(1)  Continuous  type  across  page. 

(2)  Use  of  standard  typewriter  to  prepare  reproduction  copy. 

(3)  Use  of  office-type  reproduction  equipment. 

(4)  Uniform  lettering  size  on  final  copy  not  required.  However, 
lettering  shall  not  be  smaller  than  6- point  type. 

k.  Illustrations  and  Diagrams.  The  illustrations  and  diagrams  for  users 
manuals  shall  be  prepared  under  relaxed  format  style  and  shall  be  used  In  lieu 
of  halftones  whenever  possible. 
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SOFTWARE  REQUIREMENTS  ANALYSIS  REPORT 


DESIGN  ANALYSIS  REPORT 


DI-S-3606/S-128-1 

DI-S-3606 


These  two  reports  reference  different  version  of  the  same  DIO.  Only 
DI-S-3606/S-128-1  is  approved  for  use  in  the  Acquisition  Management  System 
and  Data  Requirements  Control  List  (AMSDL).  There  are  many  things  wrong 
with  this  DID. 

1.  It  Is  written  for  the  Air  Force  and  references 
AFSCO  375-7  Is  block  9. 

2.  Block  references  DI-S-3618/S-152  and  block  10 
paragraph  l.d  references  DI-S-3605/S-127-1 . 

3.  It  does  not  define  the  differences  in  system 
development  being  reported  in  the  two  reports. 
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POSITIONAL/STATION  HANDBOOK  DI-M-3409/H-109-1 


This  DID  Is  written  for  Air  Force  use^not  joint  service  use  under 
JLC.  It  specifically  references  AFR  8-2,  AFSCM/AFLCM  375-7. 

Paragraph  3.b  references  AFH  35-1. 
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DI-S-3619/S-153  -  TECHNICAL  PERFORMANCE  MEASUREMENT  REPORT 


l 


1.  The  proposed  DID  Is  primarily  hardware  oriented  and  does  roc 
address  the  software  aspects  of  the  development. 

Replace  tiiis  document  with  attached  Software  Development  Progress 
Report  (UDI-A-21435) . 

Delete  the  Navy-specific  references. 


§ 
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DATA  ITEM  DESCRIPTION 

|>  IDtHTiriCA  TtO>l  so  n 

AQtNCY 

SoK'«t« 

*  *  *  •  •  ^ 

I  • 

J^iporc,  Software  Development  Progress  (SDPR) 

NAVY-AS 

UDI-A-2U35 

This  report  presents  che  status  of  the  software  during 
its  development. 

t 

«.  n.rt 

20  JAN' m 

*.  OfPiCt  or  HM.wtkr 

SiCtUTv 

IR-5331 

?,  AP»IIC  AT  l?S.  IK  T  KAA  tU  AT  ION|h<» 

7.1  Current  status  is  based  upon  Che  information 
provided  in  the  Software  Development  Management 

Plan  (SDMP). 

7.2  Do  not  order  this  data  item  unless  Che  Software 
Development  Management  Plan,  DID  UhI-A-21434  ia 
cited  on  the  CDRL,  DD  Form  1423. 

».  R(*flUNC(l  fM*n«a<«ry  •«  till  ■■ 
Umk  |0) 

SECNAVINST  3560.1 

V  [“*»•-  xumuhiii 


>0.  'MT»UOCNl 

10.1.  this  report  shall  provide  che  status  for  software  designed  and  documented 
In  accordance  with  3ECNAVIMST  3560.1  and  shall  include  as  a  ninimucu 

(a)  The  progress  of  the  milestones  defined  in  the  Software  Development 
Management  Plan  and  causes  for  deviations,  if  any. 


(b)  Accounting  of  functions  or  threads  that  are  in  system  analysis, 

in  simulation,  in  design,  <n  code,  in  module  test,  in  system  test, 
and  in  flight  test. 

(c)  Hardware  changes  and  discrepancies  that  affect  processing. 

(d)  Major  difficulties  anticipated  or  encountered  and  plans  to  overcome 
them,  including: 


DD 


(1)  Events  that  are  currently  behind  schedule  (or  have  antici¬ 
pated  schedule  changes),  their  effect  on  completion  of  the 
project,  and  steps  being  taken  to  remedy  schedule  delays. 

(2)  Other  information  which  defines  cause  and  effect  of  sig¬ 
nificant  changes  on  the  contract  schedule. 

(3)  Problems  which  actually  or  potentially  will  cause  deviation 
from  contractual  requirements. 


1654  '■*<  -O  :  '..’••MS.  40C0  >u!|  v;, 

<*.*•»•«•**  ■* 
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UDI-A-2  U35 


(«)  Revised  schedule  wlch  projected  completion  dates. 

(f)  Manpower  incurred  versus  planned  manpower  with  projected 
manpower  to  completion. 

‘(g)  Total  costs  incurred  versus  planned  costs  with  projected 

*  costs  to  completion. 

(h)  Core  and  timing  estimates  per  module  versus  planned  core 
and  timing. 

(i)  Actual  computer  usage  versus  predicted. 

10.2  The  report  shall  conform  to  the  following: 

(a)  All  pages,  Including  attachments  shall  be  typewritten  or  clearly 
lettered  with  non-fading  ink  on  standard  letter  size  paper  or  on 
a  standard  size  engineering  drawing  paper. 

(lb)  The  first  page  shall  be  a  titla  page  on  which  the  following  data 
shall  be  contained,  located  three  inches  from  the  top  of  the  page, 
and  two  inches  from  its  unfastened  edge: 

(1)  Type  of  report,  e.g.,  monthly,  interim,  final. 

(2)  Title  as  indicated,  on  the  data  item  description. 

(3)  Contra :t  number. 

(4)  Dates  of  the  reporting  period. 

(5)  Contractor's  name. 

Other  necessary  information  may  be  included  elsewhere  on  the  title  page. 

(c)  All  figures,  tables-,  appendices,  attachments,  etc.,  will  be 
Identified  in  the  Table  of  Contents  and  within  the  text  of 
tho  report. 

(d)  Security  classification  and  distribution  limitation  markings 
shall  conform  to  the  requirements  contain  in  the  contract  to 
which  the  status  report  applies. 

10.3  Unless  otherwise  indicated  herein,  documents  cited  in  this  block  of  tho 
issue  In  effect  on  the  date  of  invitation  for  bids  or  request  for  proposals  or 
quotations  form  a  patt  of  this  DID  to  the  extent  specified  herein. 
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18  September  1981 


LTC  Casper  H.  Klucas 
HQ,  AFB/LOEC 

Wr Ight-Patterson  AFB,  OH  1*5^31 


To:  Panel  B  Participants  of  the  JLC  Software 
Workshop  -  June  1 98 1 


Enclosed  is  the  final  report  of  the  panel  on  Hardware/Software/ 
Firmware  Configuration  Item  Selection  Criteria.  Although  a  total 
concensus  on  several  complex  Issues  was  not  achieved,  I  feel  the 
panel  attacked  the  problem  In  a  structured  manner  and  have  offered 
the  JLC  CSM/CRM  several  specific  recommendations  and  have  outlined  a 
course  of  action  for  additional  effort.  I  know  I  speak  for  Ralph 
San  Antonio  and  all  the  panel  members  In  expressing  our  gradltude  at 
being  selected  to  participate  In  a  very  Interesting  and  productive 
workshop. 

Please  call  on  us  again  for  continued  support. 

Sincerely, 

\  "V ,  n  .  i  ,■ 

•’  ')  ; .  i  l  i 

R.  A.  Maher 
Cha I rperson 
Panel  B 
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P.  Mauro 
T.  Schuman 
D.  Hartwlck 
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ATTACHMENTS : 


Draft  Report  -  Panel  B  -  Hardware/Software/Firmware  Configuration  Item 

Selection  Criteria 

1.  OBJECTIVE:  The  objective  as  defined  in  Appendix  C-l,  JLC  Charter,  was 
to  examine  the  general  problem  of  hardware/software/firmware  CI/CPCI  par¬ 
titioning  and  specification  and  develop  a  set  of  criteria  to  aid  in  the 
selection  and  documentation  process.  The  problem  has  become  exceptionally 
complex  for  microcomputers  and  computer  software  embedded  in  reprogrammable 
hardware  (firmware).  The  difficulty  lies  both  in  defining  the  "hardware 
intensive"  vs  "software  intensive"  nature  and  also  in  the  style  and  com¬ 
plexity  of  management  control  documentation.  A  secondary  objective  there¬ 
fore  was  to  recommend  an  approach  for  defining  firmware/software  categories 
and  support  documentation  requirements. 

2.0  SCOPE:  Reprogrammable  CIs  have  a  varying  scale  of  software  content 
that  range  from  simple  RCM/PROM  devices,  utilized  as  design  solutions,  never 
intended  to  be  changed  or  have  attendant  support  system  requirements;  to 
software  intensive  systems,  requiring  full  CPCI  treatment  and  very  complex 
support  systems,  tools  and  maintenance  CI/CPCI s .  The  current  CI/CPCI  allo¬ 
cation  process  does  not  recognize  the  "hardware  intensive"  or  "software 
intensive"  nature  of  firmware  CIs  nor  is  adequate  criteria  available  to 
guide  in  the  CPCI  selection  process  or  scope  the  required  support  documen¬ 
tation  and  data.  The  panel  attacked  the  problem  in  the  following  manner: 

a.  Conduct  a  top  down  analysis  of  technical  and  management  considera¬ 
tions  important  in  the  selection  of  systems  hardware,  software  and 
firmware  components. 

b.  Establish  technical,  programmable  and  management  guidelines/criteria 
for  CI/CPCI  selection  and  treatment. 

c.  Test  the  criteria  against  representative  hardware/firmware/software 
architecture  for  adequacy  and  clarity. 

d.  Define  sensible  categories  of  reprogrammable  CIs  for  treatment  of 
their  software  nature  as  less  than  full  CPCIs. 

e.  Review  recommended  firnware  DIDs  and  define  documentation  require¬ 
ments  for  the  different  reprogrammable  Cl  categories. 

The  panel  effort  was  scoped  by  first  establishing  an  initial  cut  at  tasks 
a.  and  d.  and  utilizing  the  remaining  time  for  related  tasks  b.,  c.  and  e. 
considerations. 


3.  APPROACH :  The  panel  conducted  a  general  review  session  of  selected 
reference  material  from  the  bibliography.  Several  of  the  references  have 
been  included  in  the  Appendices  for  completeness  of  this  report.  Each  panel 
member  discussed  CI/CPCI  issues,  problems  and  imr^tant  considerations  from 
their  technical /management  experience  point  of  '  iew.  Several  action  items 
and  questions  were  generated  and  groupe^.  together  as  best  matched  the  five 
task  areas  defined  in  Section  2.0  (Scope).  Two  subpanels  were  established 
in  which  one  group  (subpanel  B-2)  would  attack  the  basic  selection  criteria 
problem  (Tasks  a.,  b.,  and  c.),  while  another  subpanel  (B-l)  would  jump  in 
the  middle  of  the  "what  do-I-do-now?"  problem  of  firmware  definition  and 
documentation  (Tasks  d.,  and  e.). 

4.  DISCUSSION:  Sub  Panel  B-l  divided  firmware  into  four  general  categories 
and  reviewed  the  impact  to  MIL-STDs,  DIDs  and  CPCI  selection.  References 
used  included  receii1-  EIA  findings  and  recommendations  (Reference  8).  Dr. 
Sylvesters  (USAF/ASO)  Hardware  Intensive  Treatment  white  papers  (Reference 
9-11)  and  four  candidate  sets  of  tinware  DIDs  (Reference  4-8).  It  was  con¬ 
cluded  that  reprogrammable  CIs  could  be  sensibly  divided  into  four  general 
categories  with  CPCI  impact  as  follows: 

Category  #1  -  Software  Intensive  -  Full  CPCI  and  support  system  (Docu¬ 
mentation  tailoring  as  necessary) 

Category  #2  -  Moderate  Software  Attributes  -  Non-complex  specification 
treatment  for  definition  and  documentation.  Adequate  CPCI  and  support 
system 

Category  #3  -  Hardware  Intensive  -  Cl  with  Product  Specification  - 
identification  and  new  firmware  DID.  Partial  software  support  system 

Category  #4  -  Slightly  Reprogranwable  -  Cl  with  reprogramming  data  and 
attributes  included  in  DoD  D100/D1000  drawing  release.  Some  support 
system  definition. 

Recommendations  for  a  new  DID  or  DID  application  matrix  were  made  to 
Panel  A  in  a  working  session. 

Subpanel  B-2  reviewed  the  DRC  reconmended  selection  criteria  (Reference 
14)  and  several  of  the  source  material  references  as  a  starting  point 


Initiated  systematic  process  of  Issue  statement,  criteria  identification 
decision  flow  development  and  test  criteria  application.  The  available 
time  did  not  allow  completion  of  the  full  sequence,  but  a  baseline  set  of 
selection  criteria  were  established  as  an  excellent  continuation  point 
for  this  effort. 
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4.1  SUB-PANEL  B-l,  FIRMWARE  DOCUMENTATION 
4  1  i  Background 

We  are  experiencing  an  explosion  of  microprocessor  technology  that 
will  yield  thousandfold  improvements  in  computer  resource's  cost, 
size,  and  performance  during  each  of  the  next  two  decades.  Micro¬ 
processors  are  proliferating  at  an  exponential  rate,  and  they  are 
performing  tasks  never  before  imagined  for  military  systems.  This 
unprecedented  expansion  of  computation  embedded  in  military  systems 
has  created  an  entirely  new  set  of  management  concerns  for  today's 
military  managers  and  defense  contractors. 

In  early  applications  of  microtechnology,  the  software  aspects  of  the 
application  often  were  not  visible  to  the  military  program  manager. 
Concerns  were  expressed  over  real  and  perceived  problems  in  managing 
this  new  technology  area.  Do  you  manage  it  as  hardware,  or  software, 
or  both?  We  found  ourselves  caught  up  in  a  debate  on  definitions. 

What  is  a  “microprocessor",  “microprogram",  "microcomputer",  "micro¬ 
code",  or  "firmware"?  Something  had  to  be  done  -  but  what? 

The  military  services  determined  that  the  most  significant  problem 
lay  in  establishing  a  mechanism  for  the  control  of  the  software 
aspects,  i.e.,  firmware,  of  the  microtechnology  area.  As  an  interim 
measure,  the  Military  Services  determined  that  firmware  should  be 
handled  as  software  *nd  the  microprocessors  and  microcomputers  would 
be  controlled  as  hardware  Items. 

While  this  solution  did  solve  the  Immediate  management  and  control 
problem.  It  also  Introduced  a  potential  for  significant  growth  in 
acquisition  costs.  If  all  firmware  was  to  be  handled  as  software, 
then  must  it  not  also  be  managed  as  a  Computer  Program  Configuration 
Item  (CPC I ) ?  As  a  CPCI,  the  full  rigors  of  a  structured  development 
process  would  apply:  requirements  and  product  specifications,  design 
reviews  and  audits,  development  and  management  plans,  and  test  plans 
and  procedures.  Further  use  of  an  approved  higher  order  language  in 
developing  the  CPCI  would  be  required,  data  rights  to  the  software 
would  be  acquired,  and  the  support  tools  required  to  develop  and  test 
the  firmware  would  be  deliverable.  For  many  applications  of  firmware, 
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implementation  of  the  described  requirements  would  clearly  be  a  case 
of  "over-kill". 

The  Ain  Force  S„ stems  Command,  under  the  guidance  of  Dr.  Richard  J. 
Sylvester,  developed,  in  1979,  a  draft  policy  which  described  either 
a  "hardware  intensive"  or  "software  intensive"  classification  for 
firmware  (ref.  9  and  10).  The  intent  of  this  policy  was  to  allow  the 
identification  and  description  of  the  hardware  intensive  firmware  as 
part  of  the  hardware  documentation  hierarchy,  and  to  manage  and  con¬ 
trol  hardware  intensive  firnware  as  part  of  the  hardware;  thereby 
avoiding  the  costs  inherent  in  designating  an  item  as  a  CPC1.*  At 
the  same  time,  provisions  were  made  to  designate  an  item  as  being 
software  intensive,  and  thus  subject  to  all  the  management  and  con¬ 
trol  provisions  inherent  to  the  CPCI  designation.  The  hardware  cr 
software  intensive  classification  seemed  to  offer  a  possible  solution 
to  the  problem  of  cost  effective  management  and  control  of  firmware. 

A  problem,  of  course  exists  in  defining  what  comprises  a  hardware 
intensive  versus  a  software  intensive  application.  Technology  is 
moving  so  fast,  it  is  questionable  whetheranyone  could,  in  fact, 
define  a  set  of  classification  which  would  remain  valid  for  an 
extended  period.  Dr.  Sylvester's  most  recent  report  on  this  subject 
(ref.  11)  does  provide  a  good  discussion  of  cases  where  the  distinction 
can  be  made.  However,  many  other  cases  are  less  clear.  Figure  1 
diagrams  the  "fuzziness"  which  occurs  when  attempting  to  determine  wnat 
classification  to  assign  to  firmware.  Conceivably,  all  microprocessor/ 
firmware  applications  falling  to  the  left  of  point  "a"  might  be  ex¬ 
cluded  from  most  software  management  requirements,  and  all  applications 
to  the  right  of  point  "c"  would  be  subject  to  the  normal  software  control 
practices.  Applications  falling  between  "a"  and  "c"  would  be  considered 
a  case-by-case  basis  (ref.  12). 


♦Sylvester  Report  (ref.  11)  page  3,  "For  hardware  intensive  applications, 
policy  .  .  .  does  not  exclude  in  the  imposition  of  full  MIL-STD  documentation 
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Figure  1.  Firmware  Categories  for  Management, 
The  "Gray  Zone" 


4.1.2  Sub-Panel  Charge 

Against  the  preceding  background,  the  basic  problem  assigned  to  our 
sub-panel  was  to  determine  what  methods  currently  exist,  or  must  be 
developed,  to  establish  required  management  and  control  of  firmware. 
Both  hardware  intensive  and  software  intensive  firmware  was  to  be 
considered.. 

4.1.3  Sub-Panel  Deliberation  Summary 
a.  Assumptions 

The  sub-panel  began  their  deliberations  using  the  following  basic 

assumptions,  or  precepts: 

•  All  firmware  has  both  software  and  hardware  aspects. 

•  The  development,  release  and  maintenance  of  firmware  must  be 
controlled  following  basic  project  management  and  configura¬ 
tion  management  practices. 

•  While  the  charter  which  must  be  developed  to  determine  whether 
firmware  should  be  classified  as  hardware  or  software  inten¬ 
sive  is  extremely  important,  it  was  neither  within  the  purview 
of  the  sub-panel  to  establish  this  criteria  nor  was  it,  required 
to  accomplish  our  purpose. 


•  Maximum  advantage  should  be  taken  of  existing,  or  proposed 
specifications  and/or  Data  Item  Descriptions  (DIDs)  in  devel¬ 
oping  recommendations  for  firmware  identification  and 
documentation. 

b.  General  Discussion 

All  government  panel  participants  and  the  Mitre  representative 
expressed  significant  concern  over  the  management  of  firmware, 
and  the  need  to  provide  Increased  visibility  into  its  (firmware) 
design,  development,  test,  operation,  and  maintenance.  Each 
government  representative  was  able  to  provide  examples  of  current 
or  past  projects  wherein  lack  of  effective  control  of  firmware  or 
related  support  tools  had  led  to,  or  was  leading  to,  difficulties 
in  its  management  and  control.  Major  emphasis  was  placed  during 
these  discussions  on  the  deed  for  providing  improved  visibility 
and  control  of  the  software  aspects  of  firmware.  However,  it  was 
also  agreed  that  designation  of  all  firmware  as  CPCI's  was  not 
required  for  effective  management,  nor  was  the  CPCI  designation 
always  defensible  from  a  cost  effectiveness  standpoint.  Judi¬ 
cious  use  of  the  hardware  and  software  intensive  classification 
for  firmware  was  generally  endorsed  as  a  means  of  effecting  the 
proper  level  of  management  visibility  and  control. 

Concern  was  also  expressed  on  the  need  to  ensure  that  the  appro¬ 
priate  degree  of  firmware  support  tools  (those  tools  required  to 
develop,  test,  enhance,  and  maintain  the  firmware)  be  acquired 
as  part  of  the  basic  procurement.  The  level  of  support  tools 
required  is  closely  oriented  to  the  planned  application  of  the 
firmware  and  the  planned  maintenance  concept. 

While  the  discussion  of  support  tools,  or  environment,  is  closely 
allied  with  the  firmware  management  and  control  problem  addressed 
by  the  sub-panel,  it  was  eventually  agreed  that  the  subject  was 
not  specifically  germane  to  our  chapter. 
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The  panel's  Industry  representatives  described  their  current 
policies  and  practices  related  to  firmware.  Systems  have  evolved 
which  can,  perhaps,  be  best  described  as  being  of  a  hybrid  nature, 
e.g.,  a  combination  of  hardware  and  software  identification  and 
control  criteria  and  presses.  These  policies  and  practices 
have  been  developed  in  response  to  the  perceived  need  by  indus¬ 
try  to  properly  identify  and  control  their  microtechnology 
products  .  .  .  both  hardware  and  software.  Although  these  prac¬ 
tices  have  been  effective  for  the  firms  involved,  there  are  per¬ 
ceived  weaknesses  in  the  process: 

•  Each  firm  has  established  systems  which  are  unique  unto 
themselves, 

•  the  customer  has  no,  or  limited,  visibility  into  these  inter¬ 
nally  developed  and  implemented  policies  and  practices,  and 

•  no  convenient,  or  formal,  contractual  vehicle  exists  for  the 
government  to  request  the  firmware  data. 

There  was  considerable  discussion  of  the  adviseabil ity  of  treat¬ 
ing  the  documentation  of  the  software  aspects  of  firmware  in  the 
same  fashion  as  conventional  software,  without  the  stipulation 
that  it  be  designated  as  a  CPCI.  If  this  path  were  pursued,  the 
minimum  documentation  for  the  software  aspects  of  firmware  would 
have  to  be  established  well  below  the  full  CPCI  level.  The  con¬ 
cept  of  a  tailored  CPCI  was  introduced  during  these  discussions 
to  designate  those  diminished  documentation  requirements.  The 
concept  was  finally  dropped  by  the  panel  in  favor  of  the  recom¬ 
mendations  in  paragraph  4.1.4  but  a  minority  report  has  been 
included  in  paragraph  4.1.5  to  record  (and,  hopefully,  more 
clearly  identify)  the  issue  on  which  so  much  fruitful  discussion 
was  based. 

The  panel  also  considered  the  adviseabi lity  of  imposing  direc¬ 
tives  on  the  use  of  HOL's  for  firnware  and  on  the  use  of  specific 
computer  Instruction  Set  Architectures  (ISA's).  The  unanimous 
conclusion  was  that  the  use  of  HOL's  for  firmware  should  be 
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encouraged  but  should  not  be  made  mandatory  by  directives, 
especially  for  hardware  intensive  applications.  That  is,  the 
use  of  specific  HOL's  and/or  the  use  of  HOL's  in  general  should 
not  be  edicted.  This  conclusion  was  based  on  the  conviction  that 
such  edicts  would  often  create  costs  which  are  not  recovered  in 
the  life  cycle.  Where  the  use  of  HOL's  is,  in  fact,  practical  - 
the  designer  will  likely  elect  to  use  the  HOL  since  that  will 
simplify  his/har  task.  A  similar  conclusion  was  reached  with 
respect  to  ISA's,  although  it  was  not  unanimous  in  that  case. 

c.  The  Problem 

The  sub-panel's  deliberations  pointed  to  the  need  to  establish 
an  efficient  and  effective  method  of  identifying  both  hardware 
and  software  intensive  firmware.  Once  identified,  there  is  a 
need  to  tailor  the  acquisition  process  to  accommodate  the  unique 
characteristics  of  firrrware. 

The  sub-panel  further  expressed  the  opinion  that  current  DoD 
military  standards,  and  related  Data  Item  Descriptions  (DIDs ) , 
do  not  adequately  address  firnware  management  and  control.  Short¬ 
comings  identified  in  the  MIl-STDs/DIDs  were: 

•  MIL-STD-483/490:  Provisions  for  inclusion  of  the  software 
elements  of  firmware,  or  for  providing  traceability  to  these 
elements,  is  not  provided  in  the  Type  A,  B,  and  C  hardware 
specification  detailed  requirements  appendices.  The  reverse 
situation  is  true  for  the  hardware  elements  of  firmware  in  the 
related  software  specification  requirements  appendices.  Such 
provisions  are  required  to  support  the  respective  classifica¬ 
tion  of  firmware  as  hardware  or  software  intensive. 

•  ^IL-STD-1679:  Provisions  for  inclusion  of  the  hardware  ele¬ 
ments  of  firmware,  or  for  providing  traceability  to  these 
elements,  is  not  provided  in  the  documentation  DIDs  related 

to  this  MIL-STD.  These  provisions  are  required  to  support  the 
classification  of  firmware  as  software  intensive. 
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Similar  findings  would  also  apply  to  the  documentation  formats 
found  in  DOD-7935.1  5. 


Since  the  above  referee..??:.  specifications  are  key  to  establish¬ 
ing  the  configuration  identification  of  all  CI/CPCIs,  the  sub¬ 
panel  believed  it  was  critical  that  these  MIL-STDs/DIDs  be 
modified  to  effectively  accommodate  firmware.  Good  configuration 
management  practices  dictate  that  whenever  feasible,  a  single 
configuration  identification  (Cl  or  CPCI)  be  applied  to  firmware. 


Proposed  Documentation 


During  the  past  several  years,  the  military  Services,  industry, 
and  many  of  the  industry  associations  have  devoted  significant 
time  and  energy  in  discussing  the  firmware  issue,  and  in  develop¬ 
ing  documentation  schemes  for  firmware.  Our  sub-panel  reviewed 
several  specific  documents  which  have  been  developed  as  part  of 
this  overall  effort,  and  which  appeared  to  provide  a  possible 
solution  to  the  problems  with  the  existing  MIL-STDs/DIDs  cited 
in  c.  above. 


(1) 


Panel  D,  "Tailoring  CM  Practices  for  Microprocessors  and 
Firnware",  Electronics  Industries  Association  (EIA),  Report 


of  the  Fourteenth  Annual  Data  and  Configuration  Management 


Workshop;  October  20-24,  1980 


The  report  prepared  by  Panel  D  at  the  1980  EIA  Workshop  re¬ 
viewed  configuration  management  requirements  for  micropro¬ 
cessors  and  firmware.  Two  recommendations  made  by  this 
panel,  which  were  of  specific  interest  to  our  sub-panel: 


•  Define  a  new  element  of  a  hardware  Configuration  Item  (Cl) 
entitled  a  Computer  Program/Hardware  (CP/H).  The  CP/H  is 
microprocessor  firmware  which  is  identified  and  controlled 
as  a  portion  of  a  hardware  configuration  item.  (Appen¬ 
dices  C-3  and  C-5.) 
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•  Prepare  a  Data  Item  Description  (DID)  for  the  Computer 
Program/Hardware  (CP/H).  The  DID  would  be  in  the  format 
of  a  Book  Form  drawing  and  would  be  called  for  as  a  CDRL. 
The  DID  would  reference  DQD-STD-100  and  DOD-D-IOOO.  See 
Attachment  1  for  detail. 

(2)  UDI-E-3935,  "Firmware  Development  Plan";  September  14,  1979. 

This  proposed  DID  provides  format  and  content  guidance  for  a 
Firnware  Development  Plan  (FDP).  The  FDP  is  the  engineering 
management  plan  which  identifies  the  contractor's  efforts 
required  to  develop  and  deliver  computer  resources  and  the 
associated  firmware,  processes,  documentation,  and  necessary 
support  resources.  See  Appendix  C-6  for  detail.  The  plan 
would  be  appropriate  when  a  procurement  does  riot  require  a 
Software  Development  Plan/Computer  Program  Development  Plan 
to  be  developed. 

(3)  UDI-E-3936  —  ASO,  "Firmware  Technical  Description  (Product. 
Specification)";  March  30,  1979. 

This  proposed  DID  provides  a  complete  and  oetailed  technical 
description  of  each  functional  implementation  of  firmware. 

The  document(s)  serve  as  an  instrument  for  acceptance,  modi¬ 
fication,  trouble  diagnosis,  maintenance,  and  reprocurement 
of  firmware.  See  Appendix  C-7  for  detail. 

(4)  UDI-E-3937  -  ASD,  "Firmware  Support  Data";  March  9,  1979. 

This  proposed  DID  provides  a  description  of  the  data  required 
for  the  maintenance  and  modification  of  firmware  ROMs  (Read 
Only  Memories)  and  PROMs  (Programmable  Read  Only  Memories), 
microprocessors,  and  other  firmware  devices.  See  Appendix 
C-8  for  detail. 

(5)  Draft  AFSC/ASD  paper,  "Hardware  Intensive  Application  of 
Computer  Resources",  Dr.  Richard  J.  Sylvester;  June  4,  1901 
(Reference  #11). 
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This  paper  provides  a  full  discussion  of  hardware  intensive 
firmware:  the  advantages  and  disadvantages  which  are  in¬ 
herent  in  the  classification,  criteria  for  selection  of  the 
classification,  documentation  required,  testing,  and  SOW 
guidance.  Further,  a  draft  DID,  "Configuration  Item  Product 
Fabrication  Specification"  was  provided.  See  Appendix  C-9 
for  DID  detail. 

4.1.4  Sub-Panel  Recommendations 

The  sub-panel  recommended  the  following  actions  be  considered  by  the 
Joint  Logistics  Commanders,  Computer  Resources  Management  Committee, 
and  Computer  Software  Management  Sub-group: 

a.  Consideration  be  given  to  the  adoption  of  the  1980  EIA  Panel  D 
recommendations  for  identification  of  a  Computer  Program/Hardware 
(CP/H)  component  of  a  hardware  Configuration  Item  (Cl).  A 
related  recommendation  is  for  the  publication  of  a  DID  referenced 
to  DOD-STD-IOO-DOD-D-IOOO,  describing  the  Book  Form  drawing  which 
encompasses  the  software  elements  of  firmware  (see  Appendix  C-5). 

The  documentation  provided  under  this  DID  would  serve  to  satisfy 
required  configuration  identification  data  for  selected  current 
and  future  perceived  hardware  intensive  applications  of  firnware. 
Selection  of  this  DID  would  be  appropriate  when  minimal  visibility 
into  the  software  aspects  of  firmware  is  required. 

b.  Consideration  be  given  to  the  adoption  of  the  Configuration  Item 
Product  Fabrication  Specification  DID  found  in  Dr.  Sylvester's 
draft  paper  on  "Hardware  Intensive  Application  of  Computer 
Resources"  (See  Appendix  C*9). 

The  documentation  provided  under  this  DID  would  serve  to  satisfy 
required  configuration  identification  data  for  selected  current 
and  future  perceived  hardware  intensive  applications  of  firmware. 
Selection  of  this  DID  would  be  appropriate  when  greater  detail  and 
visibility  into  the  software  aspects  of  firmware  is  required. 
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c.  Consideration  be  given  to  the  adoption  of  the  Firmware  technical 
Description  (Product  Specification)  DID  developed  by  AFSC/ASD. 

This  DID  should  be  published  under  MIL-ST0-4G3  and/or  490  (see 
Appendix  C-7). 

The  documentation  provided  under  this  DID  would  serve  to  satisfy 
required  configuration  identification  data  for  selected  current 
and  future  software  intensive  applications  of  firmware.  Selec¬ 
tion  of  tnis  DID,  which  would  define  what  the  sub-panel  referred 
to  as  a  "tailored  CPCI",  would  be  appropriate  when  it  is  deemed 
feasible  to:  combine  the  performance,  detailed  design  require¬ 
ments,  test  and  evaluation  requirements,  and  hardware  aspects  of 
firmware  in  one  specification. 

d.  Consideration  be  given  to  the  issue  of  a  DID  referenced  to  MIL- 
STD-483,  490,  and  1679,  which  would  modify  the  software  specifi¬ 
cation  formats  contained,  or  referenced  therein,  to  include  data 
in  the  form  of  an  appendices,  on  the  hardware  aspects  of  a  soft¬ 
ware  intensive  firmware  application.  The  appendices  would  be 
similar  in  intent  to  that  which  has  been  provided  in  Appendix 
C-9,  except  it  would  be  for  hardware  versus  software. 

The  documentation  provided  under  this  DID  would  serve  to  satisfy 
required  configuration  identification  data  for  those  firmware 
applications  determined  to  be  fully  software  intensive. 

e.  The  preceding  reconmendation,  since  they  are  all  keyed  to  exist¬ 
ing  MIL-STD's,  should  be  considered  for  implementation  as  soon 
as  practicable. 

To  ensure  a  longer  term  solution  to  the  firmware  documentation 
problem,  it  is  requested  that  the  provisions  of  the  documentation 
changes  requested  in  a.  through  d.  above  be  considered  by  the 
members  of  Panel  A  when  defining  their  final  recontnended  DID  list. 


To  provide  an  overview  of  our  recommendation  for  firmware  docu¬ 
mentation,  refer  to  Figure  ?.. 
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1.  -  AFSC/ASD  Cl  Product.  Fabrication  Specification  (paragraph  A. 1.5b) 

2.  ElA  Book  Form  Drawing  for  CP/H  (Paragraph  4.1.5a) 

3.  AFSC/ASD  Firmware  Technical  Description  (paragraph  4. 1 .5c) 

4.  Sub-Panel  recommended  modification  to  MIL-STD-483,  490,  and  1679 
(Paragraph  4. I . 5d) 

Figure  2.  Recommended  Firmware  Documentation  Scheme 
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NOTF:  The  subpanel  did  not  have  sufficient  time  to  review  the 
recommended  DIDs  in  detail  (Appendices  C-6,  C-7  and  C-8), 
therefore,  a  full  review  of  the  DIDs  should  be  accomplished 
to  Insure  appropriateness  and  correctness  for  the  recom¬ 
mended  application. 

f.  It  is  further  requested  that  Panel  r's  consideration  be  extended 
to  the  DIDs  found  in  Appendix  C-6  (Firmware  Development  Plan), 
and  4  (Firmware  Support  Data).  This  consideration  should  ensure 
that  the  data  encompassed  in  thse  DIDs  has  been  provided  for  in 
their  final  recommeded  DID  list.  It  is  ojr  understanding  that 
Appendices  C-6.  C-7  and  C-4  are  intended  to  be  utilized  as  a 
package. 

g.  Do  NOT  impose  HOL  direction  on  firmware  developed  (especially 
hardware  intensive  firmware)  but  do  encourage  its  use. 
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h.  Do  NOT  impose  specific  ISA  direction  on  firmware  controlled 
computers  (especially  for  hardware  intensive  applications)  but 
do  encourage  use  of  common  ISA's. 

4.1.5  Minority  Viewpoint 

When  considered  from  the  viewpoint  of  the  software  practitioner, 
there  would  appear  to  be  no  difference  between  conventional  software 
and  firmware,  except  for  the  requirement  to  make  a  hardware  change 
whenever  the  associated  firmware  program  is  changed.  Why  then 
shouldn't  the  existing  software  documentation  technology  be  perfectly 
adequate  for  covering  the  software  aspects  of  firmware?  After  much 
discussion,  the  sub-panel  decided  not  to  pursue  this  approach  but 
rather  to  support  efforts  to  define  special  docimients  to  contain 
coverage  of  the  software  aspects  of  firmware  (see  recommendations 
above).  While  the  final  vote  on  this  issue  was  clear-cut,  the  issue 
merits  recording  here  since  it  does  not  receive  explicit  treatment 
in  the  current  literature  and  since  it  is  likely  to  elicit  support 
from  the  software  community  if  the  spectre  of  "full  CPCI"  treatment 
can  be  disspelled. 

In  the  fourth  subparagraph  of  4.1.1,  the  reasons. for  avoiding  the 
use  of  "conventional  software  documentation"  for  firmware  are  well 
presented.  We  are  all  familiar  with  attempts  to  apply  full  CPCI 
treatment  to  a  small  firmware  program  with  little  prognosis  for  change 
during  its  life  cycle.  This  certainly  is  overkill  but  it  does  not 
directly  follow  that  a  more  suitable  approach  cannot  be  found  within 
the  software  documentation  technology.  In  fact,  the  existing  MIL- 
STD's  permit  the  tailoring  of  the  CPCI  concept  to  be  a  subset  of  "full 
CPCI"  treatment.  However,  this  approach  is  not  often  used  in  prac¬ 
tice  -  perhaps  due  to  ignorance  of  the  existence  of  such  an  option. 

Many  firmware  programs  are  developed  by  design  engineers  with  a  dom¬ 
inantly  hardware  background.  Their  natural  reluctance  to  get  involved 
in  the  apparent  maze  of  "software  documentation"  is  another  reason  for 
avoiding  this  approach.  However,  if  one  believes  that  there  is  no 
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technological  difference  between  a  firtrware  program  and  a  software 
program  of  comparable  complexity,  then  avoiding  the  existing  devel¬ 
opment  and  documentation  technology  for  software  will  dcom  these  new 
firmware  designers  to  relearn  the  lessons  which  were  painfully  taught 
to  today's  software  practitioners  over  many  years  of  evolution  and 
refinement.  It  would  be  more  effective  to  introduce  them  to  the 
least  complex  forms  of  software  documentation,  which  will  likely 
prove  adequate  for  most  of  their  work. 

From  this  perspective,  the  identification  of  a  plethora  of  new  DIDs 
(as  recommended  in  4.1.4  and  elsewhere)  to  cover  the  software  aspects 
of  firmware  is  redundant,  wasteful,  and  is  likely  to  be  ineffective 
in  many  cases.  To  illustrate  the  point,  suppose  two  engineers  develop 
the  identical  program.  But  one  is  implemented  in  Read  Only  Memory 
(ROM)  and  is  considered  firmware,  while  the  other  is  implemented  in  a 
writeable  memory  and  is  considered  software.  Why  should  these  pro¬ 
grams  be  documented  in  two  different  ways?  Their  software  design 
aspects  clearly  call  for  uniform  treatment!  The  plethora  of  firmware- 
unique  DIDs  would  force  such  disparity. 

To  pursue  the  thesis:  uniform  documentation  for  programs  of  compara¬ 
ble  complexity  —  it  would  be  necessary  to  clarify  the  available 
options  for  handling  the  case  of  minimum  complexity.  A  minimum  doc¬ 
umentation  package  for  such  a  case  might  be  defined  to  be  a  Version 
Description  Document  and  i  Listing  Document  (an  approach  which  has 
been  successfully  used  by  at  least  one  manufacturer).  Sub-Panel  #1 
elected  not  to  pursue  this  path  so  no  further  specific  recomnenda- 
tions  are  available  as  a  result  of  their  deliberations. 
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4.2  SUBPANEL  B-2;  CI/CPCI  SELECTION  CRITERIA 


4.2.1  Objective.  The  objective  of  Subpanel  B-2  was  to  identify  the  tech¬ 
nical  and  management  needs  that  should  be  considered  in  selecting  the  con¬ 
figuration  items  from  the  hardware,  software  and  firrrware  components  of  the 
system. 

4.2.2  Scope.  This  effort  was  limited  to  developing  criteria  which  can  be 
used  by  a  program  manager  to  select  the  minimum  set  of  CIs/CPCIs  for  a  given 
system  considering  management,  technical,  and  programmatic  issues.  These 
criteria,  along  with  appropriate  guidelines,  will  assist  the  program  manager 
in  deciding  how  the  individual  components  of  a  system  should  be  managed. 
Outside  the  scope  of  this  panel  were  the  tradeoffs  which  can  be  made  between 
the  management  visibility  and  control  provided  by  a  given  CI/CPCI  structure 
and  the  attendant  cost.  (Subpanel  B-l  dealt  with  the  specific  issue  of  how 
these  tradeoffs  should  be  made  for  the  firmware  components  of  a  system.) 

4.2.3  Approach.  The  approach  followed  by  Subpanel  B-2  was  to:  first,  state 
the  issues;  second,  identify  the  list  of  appropriate  criteria  and  subcriteria; 
third,  develop  guidelines  (decision  flows,  text)  for  applying  the  criteria; 
and,  last,  test  the  criteria,  subcriteria,  and  guidelines  against  some  real- 
world  examples. 

4.2.4  Discussion.  CI/CPCI  designation  provides  a  level  of  management  visi¬ 
bility  and  control  identified  as  follows: 

•  Formal  configuration  identification 

•  Development/product  specifications 
§  Government  approval  of  changes 

9  Configuration  status  accounting  records 

•  Individual  design  review  activity 

•  Individual  qualification  testing 

•  Individual  physical  and  functional  configuration  audits 

•  Direct  ECP  handling 

•  Separate  identification,  marking 

•  Separate  operating  and  user  manuals 

•  Standardization  requirements. 


The  "cost"  associated  with  a  given  CI/CPCI  structure  Is  measured  not  only  in 
dollars,  but  also  in  the  administration  burden,  personnel  resources,  and 
complexity  Introduced  into  a  program  by  the  above  requirements. 

Historically,  there  has  been  very  little  guidance  available  to  acqui¬ 
sition  managers  for  selecting  the  hardware,  software,  and  firmware  configura¬ 
tion  items  of  a  system.  For  many  years  this  lack  of  guidance  resulted  in 
very  little  visibility  and  control  over  the  software  elements  of  the  system 
as  the  software  was  treated  as  data  (a  disposable  cormodity)  procured  in 
conjunction  with  the  hardware  elements  of  the  system.  This  period  was  fol¬ 
lowed  by  one  in  which  there  was  a  tendency  to  overspecify  the  number  of 
CPCIs  (to  maximize  visibility'  and  control),  which  resulted  in  excessive  cost 
and  administration  burden,  increased  program  complexity,  and  a  drain  on  per- 
sennel  resources.  The  net  result  was  to  decrease  visibility  and  control 
over  the  software. 

A  major  step  forward  was  achieved  with  the  addition  of  Appendix  XVII  to 
MIL -STD-483.  This  appendix  identifies  the  importance  of  and  provides  guide¬ 
lines  for  proper  CI/CPCI  selection:  it  was  the  starting  point  for  the  work 
undertaken  by  this  subpanel.  Other  reference  documents  reviewed  by  the  sub¬ 
panel  are  identified  in  Table  4.2-2.  These  references  were  used  to  develop 
the  "strawman"  list  of  CI/CPCI  selection  criteria  contained  in  Table  4.2-1. 

Subpanel  B-2  identified  the  following  issues  related  to  CI/CPCI  selec¬ 
tion  criteria: 

•  For  a  given  system,  what  are  appropriate  criteria  for  selecting 
the  configuration  items  from  the  hardware,  software,  and  firmware 
components  of  the  system?  or 

•  When  is  an  item  important  enough  to  be  considered  a  configuration 
item? 

To  address  these  issues,  the  subpanel  developed  subcriteria  and  guide¬ 
lines  for  each  of  the  criteria  listed  in  Table  4.2-2.  (See  Appendix  C-4.) 
These  are  contained  in  the  following  sections.  In  practice,  the  acquisition 
manager  would  apply  each  criterion  to  the  program  element  under  considera¬ 
tion,  weigh  the  relative  importance  between  criteria,  and  decide  if  the  pro¬ 
gram  element  merits  treatment  as  a  CI/CPCI,  "tailored"  CI/CPCI,  or  as  a 
component  of  one  of  the  above! 


Table  4.2-1.  Panel  B  Reference  Material 


MIL-STD-483 

March  21,  1979 

Configuration  management  practices  for 
systems,  equipment,  munitions,  and  com¬ 
puter  programs 

ESD-TR-77-254 

August  1977 

An  Air  Force  guide  to  computer  program 
configuration  management 

AFSCP  800-7 

December  1,  1977 

Configuration  management 

UDI-E-3935 

September  14,  1979 

Firmware  development  plan  data  item 
description  (DID) 

UDI-E-3936 

March  30,  1979 

Firmware  technical  description  (product 
specification)  DID 

UDI-E-3937 

March  9,  1979 

Firmware  support  data  DID 

EIA  G-33 

Panel  D  Report 

October  1980 

Tailoring  CM  practices  for  micropro¬ 
cessors  and  firmware 

Table  4.2-2.  CI/CPCI  Selection  Criteria 

Criticality 

Function 

Maintenance  Concept 

Supplier 

Interfaces 

Use 

Location 

Size 

Schedule/Phasing 


4.2.4. 1  CI/CPCI  Selection  Criteria.  The  information  contained  in  this 
section  is  intended  to  assist  program  managers  in  deciding  how  the  indivi¬ 
dual  components  of  a  system  should  be  managed.  Detailed  guidelines  for  the 
following  criteria  are  provided  in  order:  location,  criticality,  function, 
maintenance  concept,  use,  supplier,  interfaces,  size,  and  program  schedule/ 
phasing.  To  use  this  information,  the  program  manager  should  apply  each 
criterion  to  all  program  elements  under  consideration,  weigh  the  relative 
importance  between  criteria,  and  decide  If  each  element  merits  treatment  as 
a  CI/CPCI,  "tailored"  CI/CPCI,  or  as  a  component  of  one  of  the  above. 

4. 2. 4. 1.1  Critical i ty .  If  failure  of  the  element  would  adversely  affect 
security,  human  safety,  the  accomplishment  of  a  mission,  or  nuclear  safety, 
or  would  have  a  significant  financial  impact,  strongly  consider  identifying 
the  element  as  a  separate  CI/CPCI. 


4. 2. 4. 1.3  Maintenance  Concept.  When  different  agencies  have  responsibility 
for  maintaining  parts  of  an  element,  the  program  manager  should  consider 
breaking  the  element  into  separate  Cl /CPC Is . 

When  the  source  of  an  element  will  maintain  it,  or  when  it  is  intended 
that  the  element  be  replaced  rather  than  repaired,  the  program  manager  can 
safely  consider  tailoring  the  CI/CPCI  or  including  the  element  as  part  of  a 
larger  CI/CPCI.  Additionally,  the  support  tools  need  not  be  identified  as 
CI/CPCI  to  be  delivered. 


4.2.4. 1.4  Supplier.  Elements  provided  by  different  suppliers  should  be 
assigned  to  separate  CI/CPCIs. 
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4. 2.4. 1.5  Interfaces.  Interfaces  among  CIs  and  CPCIs  should  be  simple. 
Functions  which  are  highly  data  or  control  interdependent  should  be  allo¬ 
cated  to  the  same  CI/CPCI.  Functions  which  exhibit  a  high  disparity  between 
input  and  output  data  rates  should  be  allocated  to  separate  CI/CPCI s . 


4.2.4, 1.6  Use.  Elements  which  are  general  purpose  in  nature,  require  the 
capability  to  be  operationally  reprogrammed,  or  are  intended  to  be  reused 
in  another  system  should  be  considered  as  separate  CI/CPCI s. 


4. 2. 4. 1,7  Location.  The  functions  allocated  to  a  CI/CPCI  should  not  be 
partitioned  among  separate  geographic  areas.  Functions  allocatei  to  physi¬ 
cally  distinct  processors  in  a  distributed  environment  should  be  considered 
as  separate  CI/CPCIs. 


4. 2. 4. 1.8  S-ize.  CI/CPCI  selections  which  cannot  be  made  on  the  basis  of 
other  criteria  should  be  made  to  keep  the  CI/CPCI  to  manageable  proportions. 

4. 2. 4. 1.9  Schedule/Phasing.  Clements  scheduled  for  development,  testing, 
and  delivery  at  different  times  should  be  assigned  to  separate  CIs/CPCIs. 

4.2.4.1.10  Criteria  Development  Status.  Although  the  subpanel  textual 
descriptions  of  how  the  criteria  and  subcriteria  should  be  applied  to  a 
given  program  element,  there  was  insufficient  time  to  develop  decision  flow 
diagrams  which  would  lead  the  acquisition  manager  through  the  CI/CPCI  selec¬ 
tion  process.  In  addition,  the  subpanel  was  not  able  to  vigorously  test  the 
criteria,  subcriteria,  and  guidelines  against  actual  system  configurations. 

4.2.5  Recommendations.  Subpanel  B-2  recommends  that  the  JLC-CSM  subgroup, 
support  continued  work  in  this  area  to  accomplish  the  following: 

1.  Expand  and  refine  selection  criteria  and  accompanying  guidance 

2.  Test  criteria  and  guidance  against  actual  system  configurations. 

3.  Document  expanded  guidance  in  existing  and  planned  acquisition 
management  guidebooks. 

4.  Revise  MIL-STDs  to  include  new  requirements  as  appropriate. 

Coordinate  the  effort  in  recommendations  closely  with  the  on-going 
JLC  documentation  and  standards  development  activities. 


5. 


5.0  Panel  8  Recommendations 

The  following  are  general  recommendations  based  on  Panel  B  review  of  the 
C/1/CP/C1  selections  criteria  problem.  Detailed,  specie  recommendations  are 
contained  in  Subpanel  B-l  and  B-2  reports  (Silicons  4.1.4  and  4.2.5). 

1.  The  JLC  should  recognize  and  adopt  a  po'icy  of  reprogrammable  C.I. 
categories  and  take  appropriate  action  to  revise  MIL-STD-490,  483,  480  and 
DOD-STD-IOO/D-TOOO  necessary. 

2.  JLC  obtain  a  contractive  effort  to  refine  the  firmware  treatment 
categories  further  develop  selection  criteria; establish  test  criteria,  cases 
and  test  architectures;  and  perform  allocating  during  system  aquisitions 
activities. 

3.  Panel  A  and  Panel  B  meet  as  a  follow-on  activity  to  jointly  review 
and  develop  a  matrix  DID  firmware  appliciabil ity  matrix  and/or  review 
proposed  firmware  DIDs  (Appendix  C-6  thru  C-9). 
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1.  Rationale:  From  the  government  viewpoint,  few  criteria 
are  available  for  the  selection  of  computer  software  config¬ 
uration  items.  tolL-STU-483  (USAFJ,  Appendix  XVII  provides 
some  top  level  guidance  for  configuration  item  (CIJ  selection. 
The  Air  Force  Systems  Command  Pamphlet  (AFSCP)  800-7  provides 
some  limited  criteria,  primarily  as  do  not  s ,  for  selecting 
computer  software  Cls.  Various  gui deTjooks  have  also  been 
developed  by  the  iervices  which  discuss  computer  software 
selection  but  pro/ide  few  criteria  to  guide  selection. 

With  the  advent  o  microcomputers  and  computer  software  embedded 
in  firmware,  it  his  become  more  difficult  to  predetermine 
how  Cls  (for  hardware)  should  be  distinguished  from  computer 
software  Cls.  Wh.it  is  needed  is  a  complete  set  of  criteria 
for  guiding  the  selection  process.  These  criteria  should 
take  into  account  both  technical  and  management  considerations. 
In  addition,  the  criteria  must  consider  procurement,  developer 
support,  and  user  needs.  Criteria  roust  be  specified  which 
allow  for  reasona  ileness  in  selection  so  that  overhead  costs 
such  as  documentation  and  control  mechanisms  are  not  restric¬ 
tive  in  getting  tne  software  developed. 

2.  Approac ..  A  panel  should  be  formed  to  address  the  issues 
associated  with  computer  software  Cl  selection  and  to  develop 
a  set  of  criteria  to  aid  the  selection  process. 
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170 .  Guide  for  Selecting  Conf iguration  Items  (CIs) 

170.1  Purpose .  This  appendix  provides  guidance  to  government  and 
contractor  personnel  responsible  for  selecting  CIs.  As  used  herein, 
CIs  also  encompasses  Computer  Program  CIs  (CPCIs). 

170.2  Scope.  The  criteria  of  this  appendix  shall  be  used  in  the  Cl 
selection  process  whenever  it  occurs  during  the  life  cycle,  however, 
the  most  beneficial  results  from  its  application  will  be  realized 
when  used  at  the  beginning  of  the  acquisition  cycle. 

170.3  Applicability.  Each  contractor  to  the  government  shall  be 
responsible  for  his  compliance  with  this  appendix  as  well  as  the 
compliance  of  his  subcontractors,  vendors,  and  suppliers  in  accord¬ 
ance  with  paragraph  1.3  of  this  standard. 


170.4  General  Considerations. 

170.4.1  Selection  of  CIs  is  based  on  the  definition  contained  in  DODD 
S0 10. 19,  "an  aggregation  of  hardware/software,  or  any  of  its  discrete 
portions,  which  satisfies  an  end  use  function...  CIs  are  those  speci¬ 
fication  items  whose  functions  and  performance  parameters  must  be 
defined  an:'  controlled  to  achieve  the  overall  end  use  function  and 
performance. " 

170.4.2  The  selection  of  CIs  is  normally  a  function  of  anticipated 
design  and  should  be  independent  of  the  concept  for  future  reprocure¬ 
ment.  The  selection  process  is  one  of  separating  the  elements  of  a 
system  into  individually-identified  sub-sets  for  the  purpose  of 
managing  their  development.  The  Cl  should  be  regarded  as  a  deliver¬ 
able  entity  to  which  certain  system  functions  have  been  allocated. 

Cl  selection  reflects  an  optimum  management  Israel  during  acquisi¬ 
tion.  This  level  is  one  at  which  the  procuring  activity  specifies, 
contracts  for,  and  accepts  individual  elements  of  a  system. 

170.4.3  The  selection  of  items  to  be  managed  as  CIs  should  be 
determined  by  the  need  of  the  government  to  control  an  item's 
inherent  characteristics  or  to  control  that  item's  interface  with 
other  items.  The  selection  is  a  management  decision  normally 
accomplished  through  the  system  engineering  process  in  conjunction 
with  configuration  management  and  with  the  participation  of  logistics. 
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Selecting  Cls  should  be  with  a  full  view  of  the  life  cycle  cost 
and  management  impacts  associated  with  such  a  designation. 

Choosing  too  many  Cls  increases  the  cost  of  control;  choosing 
too  few  or  the  wrong  elements  as  Cls  runs  the  risk  of  too  little 
control  through  lack  of  management  visibility.  "Wh.it  .loos  program/ 
project  management  need  to  know  and  control?"  The  government  most 
determine  what  control  it  needs  to  exercise  in  light  of  cost/benefit 
trade-offs.  The  Cl  selections  are  made  accordingly. 

170.4.4  On  development  programs  for  subsystems  or  support  equipment 
that  will  be  common  to  more  than  one  system,  the  basic  Cl  should  be 
that  assembly  that  is  common  to  all  applications.  An  assembly  part 
that  is  required  to  meet  interface  or  other  requirements  peculiar  to 
one  of  the  systems  should  be  identified  as  a  separate  Cl  in  that 
system. 

170.4.5  The  major  elements  comprising  the  system  should  be  identi¬ 
fied  as  Cls  during  the  Demonstration/Validation  Phase.  Early 
selection  of  Cls  is  important  since  management  emphasis  becomes 
greater  as  development  progresses.  As  development  continues  and 
logistic  or  technical  considerations  surface,  additional  items 

can  be  designated  Cls.  Usually,  the  Cl  selection  process  should 
be  essentially  complete  by  PDR. 

170.5  Specific  Considerations.  The  following  are  some  of  the 
considerations  upon  which  the  decision  shall  be  based: 

170.5.1  Level  of  Government  Control.  The  Cl  must  be  a  manageable 
level  of  assembly.  The  Cl  is  the  basic  element  for  configuration 
control  and,  for  example,  is  usually  limited  to  major  subsystem 
levels  of  the  Work  Breakdown  Structure  or  to  a  critical  item  of  a 
lower  level,  when  so  identified. 

170.5.2  Engineering  Release  System.  The  Cl  must  allow  the  contractor 
to  release  engineering  changes  at  an  assembly  level  which  is  report- 
able  and  which  enables  verification  of  change  incorporation,  i.e., 
does  not  preclude  change  incorporation  verification  in  a  lower  level 
assembly . 

170.5.3  Safety.  If  the  operation  of  an  item  is  critical  to  other 
operations,  flight  safety  or  ground  safety,  the  item  will  be  more 
susceptible  to  being  classified  as  a  Cl. 

170.5.4  Existing  or  modified  existing  design  items.  Existing  items 
that  are  not  Cls  developed  at  government  expense,  should  not  generally 
be  candidates  for  reidentification  as  new  Cls  on  new  programs. 
Existing/modified  design,  commercial  off  the  shelf  eguipment/computer 
program(s),  should  not  necessarily  be  excluded  from  Cl  selection. 

The  considerations  identified  in  paragraph  170.5  and  its  subpara¬ 
graphs  should  be  addressed  prior  to  making  a  decision. 
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170.5.5  Now  or  modified  design.  Careful  consideration  shall  bo 
given  new  or  modified  design  items,  wherein  more  than  a  modest 
degree  of  complexity,  utilization  of  new  materials,  processes  or 
technology  is  involved;  and,  where  the  government  wants  direct 
control  over  the  performance  requirements  for  that  item,  at  a 
specific  time,  i.e.,  then  the  government  is  directly  concerned 
with  the  detail  development. 

170.5.6  Interface  with  GFE.  The  higher  the  degree  of  interface 
with  Government  Furnished  Equipment  (GFE),  the  higher  the  likeli¬ 
hood  for  selection  as  a  Cl. 


170.5.7  Susceptibility  to  change.  The  higher  the  anticipated  or 
estimated  degree  of  change  or  modification  which  might  be  expected 
after  the  item  is  operational  the  higher  the  likelihood  for  selec¬ 
tion  as  a  Cl. 


170.5.8  Repairability/Maintalnability .  An  item  which  is  clearly 
designated  as  "Repairable"  is  much  more  a  Cl  candidate  than  one  which 
is  not  repairable.  Eventually  logisticians  must  deal  with  the  Line 
Replaceable  Units  (LRUs)  which  comprise  the  principal  components  of 
the  subsystem.  However,  designating  CIs  at  the  LRU  level  at  the 
onset  of  full  scale  engineering  development  ( FSED )  would  add  signifi¬ 
cant  cost  to  the  development  effort,  especially  in  the  area  of 
change  management.  The  LRU  level  is  usually  too  low  a  level  for 
affective  government  control  during  development. 


170. 5. 9  Support  Equipment  Considerations.  Without  proper  planning, 
minor  items  of  support  equipment  could  swell  the  list  of  CIs. 

Minor  in  this  context  refers  to  items  such  as  individual  hand  tools, 
as  compared  to  hydraulic  torque  wrenches,  engine  build-up  tools, 
etc.  There  will  usually  be  little  or  no  change  activity  on  many 

of  these  minor  items.  It  may  be  sufficient  to  list  these  items 
as  "support  equipment  in  paragraph  3.2.4  of  the  Cl  Part  I  specifi¬ 
cation  per  MIL-STO  490,  paragraph  20. 3. 2. 4. c. 

170.5.10  subassembly  Characteristics.  Subassemblies  (within  a 
Cl)  should  have  a  common  mission  relationship;  should  have  common 
installation  and  deployment  requirements  (ground  and  airborne 
segments  would  be  separate  CIs);  should  have  a  cycle  of  changes 
dependent  on  the  Cl;  and  should  not  be  the  subject  of  separate 
test  or  formal  acceptance  by  the  procuring  activity  (should  be 
accomplished  as  part  of  a  Cl).  If  these  conditions  are  not  met, 

the  subassembly  should  be  either  part  of  another  Cl  or  a  separate  Cl. 

170.5.11  Computer  Program  Cl  (CPCI )  Considerations.  CPCI  selec¬ 
tion  In  usually  a  technically  driven  declnlon  made  by  the  developer. 
The  decl n I <>n( ;i )  I ;i /are  honed  ti|w>n  nynletn  I  rnde-nt In  and  I  he  natural 
decomposit Ion  of  the  noftwaro.  Premature  partitioning  munt  be 
avoided  because  to  a  certain  extent  it  may  preordain  the  design. 
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Generally,  the  executive/superviaor,  functional/applicationo, 
input/output,  test  and  support  programs  should  be  Individual 
CPCls.  CPs  with  potential  use  lr.  multiple  systems  should  be 
separate  CPCls,  Highly  Interrelated  CPs  should  be  combined  as 
one  CPC  1 .  CPCls  should  he  established  to  their  largest  functional 
element  (i.e.,  operational  Plight  Program,  Plight  Simulator  CP, 
etc, ) , 

170.5.11.1  Source  of  Assemblies.  Assemblies  of  CP  elements  to 

be  acquired  from  a  single  contractor  are  potentially  a  single  CPCI. 
(Separate  sources  usually  supply  separate  CPCls  for  contracting 
and  delivery  purposes. ) 

170.5.11.2  Separate  Applications.  CPs  to  be  designated  for 
operation  in  different  models  of  computers  should  be  separate 
CPCls.  Separate  CPCls  may  also  be  indicated  for  computer  programs 
when  a  given  installation  uses  a  number  of  computers  of  the  same 
type/model,  each  performing  different  functions  in  the  system  as  a 
whole  and  having  different  sets  of  interfaces  with  other  system 
elements. 

170.5.11.3  Separate  Schedules.  CPb  scheduled  for  development, 
testing,  and  delivery  at  different  times  may  i»e  separate  CPCls. 

When  indicated  by  interrelationships  and  intended  use,  however, 
consideration  should  be  given  to  such  alternatives  as:  expansion 
of  the  earlier-developed  CPCI  via  ECPi  or  development  of  the  later 
CPCI  to  incorporate  and  replace  the  earlier  item. 

170.5.12  Types .  If  there  are  different  configurations  due  to 
different  adaptation  data  for  each  operating  location,  the  different 
configurations  should  be  identified  by  types  ( MIL-STD-490 ,  para¬ 
graph  4.1.2  and  4.3b)  within  a  single  Cl. 


« 


170.6  Effects  of  Cl  Selection.  Cl  selection  affects  cost,  schedule 
and/or  performance  for  the  government,  prime  contractors,  sub¬ 
contractors  and  suppliers.  The  effects  of  Cl  selection  should  not 
be  permitted  to  occur  automatically  upon  selection  of  an  item  as 
a  Cl.  The  effects  which  are  unnecessary  or  premature  can  be 
tailored  out  for  each  CX  by  means  of  an  appropriate  contractually 
recognized  vehicle,  e.g..  Program  Plan,  Statement  of  Work,  CM 
Plan,  Exceptions  and  Deviations.  Selection  of  an  item  as  a  Cl  for 
manageability  may  be  based  on  its  administrative  complexity, 
technical  (engineering)  criticality  or  maintenance  (logistics) 
criticality.  The  following  is  a  listing  of  the  usual  effects  of 
Cl  designation: 

a.  Formal  preparation  of  discrete  configuration  identi¬ 
fication  -  most  often  in  the  form  of  a  specif ication( s ) . 
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b.  A  discrete.  development  specification  and  a  companion 
product  specification. 

c.  Government  approval  of  changes  over  the  configuration 
identification  governing  the  item. 

d.  Continuing  and  accurate  recording  of  the  exact  config¬ 
uration  status  of  the  Cl,  including  providing  field 
activities  precise  data  dealing  with  impending  or 
completed  modification  actions. 

e.  Providing  traceability  of.  detailed  design  for  follow-on 
activity,  including  historical  data  and  individual 
status  information  for  accident  investigations,  failure 
analysis,  etc. 

f.  Individual  design  review  activity  (PDR,  CDR,  FQR,  etc.) 
during  development. 


g.  Individual  qualification  testing  and  reporting. 

h.  Individual  physical  and  functional  audits  (PCA  and  VCK) 
at  the  conclusion  of  development. 

i.  Discrete  and  separate  "related"  F.CP  development  prepara¬ 
tion,  roview,  approval  and  negotiation  (for  changes  to 
CIs) . 

j«  Separate  identification  indices  and  qualification  records. 

k.  Separate  nameplates  and  discrete  Cl  identifiers  (i.e., 

Cl  number,  type,  model,  series,  etc.). 

l.  Preparation  of  separate  operating  and  user  manuals. 

m.  Too  many  CIs  may  result  in  effects  ^"hampering  visibility 
and  management  rather  than  improving  it.  These  effects 
include: 

(1)  Increased  administrative  burden  in  preparing, 
processing,  and  status  reporting  of  engineering 
changes  which  tends  to  be  multiplied  by  the  number 
of  CIs. 

(2)  Increased  development  time  and  cost  as  well  as 
possibly  creating  an  inefficient  design. 

(3)  Possible  increase  in  management  effort,  difficul¬ 
ties  in  maintaining  coordination  and  unnecessary 
generation  of  paper  work. 
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n.  Too  faw  CIS  May  result  in  costly  logistics  and  Min- 
tananca  dit f iculties.  Tha  following  May  rasult: 

(1)  Loss  of  identity  through  separation  of  affected 
portions  of  a  Cl  during  field  or  depot  mainten¬ 
ance  or  Modification  installation  activity,  i.e., 
an  autocollimator  and  its  Mount. 

(2)  Inability  to  control  like  individual  remove/ 
replace  items  when  Cl  identification  and  control 
is  at  the  "set"  level,  e.g.,  a  storage  battery 
set. 

(3)  Loss  of  operational  use  of  one  function  because 
required  maintenance  on  another  function  requires 
action  against  the  Cl  level,  e.g.,  a  Cl  having 
separate  VHF/UHF  functions  loses  both  when  Main¬ 
tenance  must  be  done  on  either  function. 

170.7  Cl  Selection  Checklist.  The  following  questions  should 
be  used  in  selecting  CIs  tailored  to  individual  program/project 
requirements.  If  most  of  the  questions  can  be  answered  NO,  the 
item  probably  should  not  be  a  Cl.  If  most  questions  can  be 
answered  YES,  the  item  probably  should  be  a  Cl.  If  the  questions 
can  be  answered  with  approximately  equal  numbers  of  YESs  and  NOs , 
additional  judgment  is  needed  to  determine  if  the  item  should  be 
a  Cl.  The  selection  of  CIs  is  a  management  decision  based  on 
experience  and  good  judgment.  It  should  be  kept  in  mind  that  some 
of  the  factors  such  as  serialisation  and  nameplates  will  be  required, 
regardless  of  Cl  selection,  e.g.,  part  of  a  higher  level  assembly. 

a.  Is  it  a  critical  high  risk,  and/or  a  safety  item? 

b.  Is  it  readily  identifiable  with  respect  to  cise,  shape 
and  weight  (hardware)? 

c.  Is  it  newly  developed? 

d.  Does  it  incorporate  new  technologies? 

e.  Does  it  have  an  interface  with  har<hiare  or  software 
developed  under  another  contract? 

f.  With  respect  to  form,  fit  or  function,  does  it  inter¬ 
face  with  other  it emu  whose  configuration  is  controlled 
by  other  entities? 

g.  Is  there  a  requirement  to  know  the  exact  configuration 
and  status  of  changes  to  it  during  its  life  cycle? 
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Panel  D  explored  alternatives  in  managing  the 
production  configuration  of  firmware  and  micro- 
proLtiSor  software.  Concerns  were  raised  with 
respect  to  the  implementation  of  programs  in 
several  read-only-memory  devices,  the  type  of 
airplane  programs  are  installed  in,  and  the  mis¬ 
sion  assigned  to  each  airplane.  As  a  result  of 
these  concens,  it  was  concluded  that  any  change 
to  seftwa-e  in  read-only-memory  devices  will  be 
treated  as  a  functional  change.  The  rationale 
for  this  approach  is  based  on  the  fact  that  all 
software  changes  require  some  retest.  It  was 
further  concluded  that  read-only-memory  device 
cnanges  will  be  identified  and  handled  in  accor¬ 
dance  with  CoD-STD-lOOC.  To  accomodate  these 
conclusions,  the  logistical  asoects  of  read-only- 
memory  devices  need  to  be  reviewed  and  considered 
as  a  oart  of  the  Preliminary  Design  Review  (PDR). 

To  form  a  basis  for  the  panel's  deliberations, 
the  followin>.  definitions  were  adopted. 

Firmware  ' s  :c  .uuter  software,  resident  in  hard¬ 
ware  read-or.ly-i.iemory  devices,  that  cannot  be 
modified  under  program  control. 

Microprocessor  Software  is  compute-  software1 
used  in  conjunction  with  micror*ac*ssor« .  It  way 
be  oelivered  as  firmware  or  as  code  aM  data 
stoi ed  on  media  (tapes,  disks,  "ard  decks,  etc.). 
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D-3 

Configuration  Aciujnting 

D-4 

Configuration  Audits 

D-5 

Library  Controls 

A  su.smary  of 

the  bacxground  information  presenta- 

tior,  and  the 

report  of  eacn  subpanel  is  provided 

hereinafter. 
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computer  Proqram/hardware  (CP/Hl  is  microproces¬ 
sor  f fr.nware  TdentYfiec  ant,  crntrolled  as  a  por¬ 
tion  of  j  ejrdware  configuration  Item. 

Prior  to  the  start  of  dei -t  rations  a  presented 
tior.,  which  provided  background  information  on 
microprocessor  software,  was  made  to  Panql  0 
participants.  Subsequent  to  this  presentation, 
thi  ..articipanCs  were  subdivided  into  five  sub¬ 
panels;  namely: 


1. Computer  software  is  n  collection  of  associated 
computer  proorams  and  computer  data  required  to 
enable  the  computer  equiprn>iit  to  perform  compu¬ 
tational  or  control  functions.  NOTE:  It  is  the 
abstract  of  tares,  disks,  card  decks,  and  firm¬ 
ware  (Reference  E1A  Bulletin  4A,  dated  April  1979). 
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The  increased  application  of  software  in  micro¬ 
processor  read-only-memory  devices  (comonly 
referred  to  as  firmware)  is  having  a  major  im¬ 
pact  on  defense  systems.  As  more  and  more 
microprocessors  and  associated  firmware  appear 
in  devices  such  as  displays,  controllers,  etc., 
configuration  management  practices  need  to  be 
reviewed  and  -evised,  as  necessary,  to  accommo¬ 
date  management  and  control  requirements  of  the 
acquisition  an  ;  support  environment.  This  is 
the  charter  of  Panel  C;  Tailoring  CM  Practices 
fcr  Microprocessors  and  Firmware. 

Imposing  extensive  software  conf iguration 
management  procedures  on  microprocessor  firm¬ 
ware  could  prove  tc  bs  unnecessarily  expensive. 
Fcr  exarole,  it  is  hard  to  visualize  treating 
a  small  computer  program,  which  is  hardwired 
(i.e..  stored  in  a  read-only-memory  device) 
into  a  display  terminal,  as  one  would  treat  a 
mere  »ophi st icateo  Computer  Program  Configura¬ 
tion  Item  (CPC! ) .  The  costs  associated  with 
conducting  cormrehensive  reviews  .audits ,  testing, 
aid  documentations,  using  such  an  approach,  could 
prove  to  ne  more  costly  than  the  development  cost 
•'or  the  small  computer  program. 

Assuming  that  the  display  terminal  is  a  hardware 
Configuration  Item  (Cl),  it  is  more  cost 
effective  to  treat  the  computer  program  as  a 
part  of  the  display  terminal  Cl  and  not  as  a 
separate  CPC1. 

On  the  other  hand,  treating  a  larger  hardwired 
computer  program  as  part  of  a  hardware  Cl  is 
pi  drably  not  a  prudent  move  either,  This  is 
certainly  the  case  if  this  program  is  expected 
to  change  once  the  Cl  is  fielded.  Changes  to 
fielded  hardwired  software  imolies  a  capability 
to  generate  a  changed  computer  program,  support 
facilities  to  b-rn-in  and  verify  these  changes 
int?  rcad-onlv-memory  devices,  and  procedures  to 
idrntify  these  channes  in  the  hardware  Implemen¬ 
tation  env iron-net. t.  When  changes  are  anticipa¬ 
ted,  there  may  aV'.o  be  need  for  a  requirement 
to  p-’ogrcm  the  software  in  a  higher  order  lan¬ 
guage  to  facilitate  the  change  process.  All  of 
th:  implies  that  sor.*  hardwlied  microprocessor 
softwiro  s(.0',» c  be  treated  as  a  CPC  I  with  the 
.attendant  rev-ews,  audits,  testing,  and  documen¬ 
ts*  Ipn.  , 


The  overall  governing  directive  within  DoD  for 
computer  resources  is  DoDD  5000.29,  "Management  of 
Computer  Resources  in  Major  Systems".  This  direc¬ 
tive  stipulates  that  computer  software  will  be 
specified  and  treated  as  configuration  items.  In 
a  recent  draft  rewrite  of  5000.29,  additional  words 
were  added;  namely:  "A  hardware  implementation  of 
computer  software  (so  called  firmware)  may  be  sub- 
jest  to  configuration  management  as  either  hard¬ 
ware  or  software  dewending  on  the  need  for  post- 
development  support". 

The  Standards  covering  software  configuration 
management  't.g.,  MIL-STD-A33,  MIL-STD-490,  etc.) 
are  silent  concern inc  the  treatment  of  microproces¬ 
sor  firmware.  However,  since  firmware  is,  in  fact, 
computer  software  that  is  hardwired,  one  can 
assume  that  the  controls  utilized  for  computer 
programs  would  equally  apply  to  firmware. 
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0-1:  IDENTIFI^ATIOM 


Subpanel  member s  were  as  follows: 

Anderson,  Jack  1. 

Bryan,  William  t. 

Cardin,  L.  P.  Roland 
*  Egan,  Leo  G. 

Ellison,  Richard 
Finch,  Frederick  0. 

Hill,  Charles  V 
Kolsti,  Heston  G. 

Lewis,  Joseph  L. 

Liberatore,  Domenic 
Lillo,  Clifford  L. 

Wallin,  Joe  0. 


*  Subpanel  Chair 


The  objectives  of  this  subpanel  were  as  follows: 

(a)  Prepare  a  typical  specification  tree  and 
design/test  documents. 

(b)  Prepare  a  flow  chart  of  the  activities  re¬ 
quired  to  achieve  microprocessor  software. 

(c)  Prepare  checklists  tha*  can  be  used  to  as¬ 
sure  the  achievement  of  configuration 
identification  objectives  for  microproces¬ 
sor  software. 

The  panel  examined  documentation  approaches  for 
micr' processor  software  when  it  is  treated  as  a: 

(a)  Configuration  Item  (Cl), 

( b )  Computer  Program  Configuration  Item  (CPCI), 
or 

(c)  Software  within  a  hardware  Cl. 

When  treated  as  a  Cl  there  was  no  problem.  The 
identification  of  a  Cl  would  be  documented  ’  via  a 
set  of  specifications,  typically  a  Typo  Sl/Ci  or 
n?/C?  Thr-  iypp  C  specification  wnuld  call  out 
a  top  iVSMir.lity  il-.iw  I  no  ,  whuii  wou'd  iilrultfy  all 
of  fac  detail  ri,)int.’priini  drawings.  When  micro¬ 
processor  software  is  treated  at  a  CP/U,  no  ~p- 
parent  orchlems  result.  Ti.e  ident if icat .bi>  of  a 
CPv.1  would  be  through  the  E S/C 5  set  of  specifice- 
tion',.  When  microprocessor  software  i  *  treated 
as  software  within  a  hardware  Cl,  many  problems 
arise.  This  is  a  "fi  s-iwace"  pi^blem.  As  an  aid 
to  resolving  this  problem,  the  microprog-iffl  or 
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software  within  a  microprocessor  device  was  called 
CP/H  (Computer  Frogram/Hardware). 

In  keeping  with  the  objectives  stated  earlier,  we 
developed  a  specification  tree  which  is  depicted 
In  Figure  1 :1  He  plso  developed  documentation 
trees  lor  the  desigr/t.est  documents  of  a  fP/H. 

These  arc  shown  in  Figures  1  1-3,  arg  1-4.  A 

time  line  chart,  tor  toe  typical  events  that  Mtuld 
take  pluct  ir,  vhe  development  of  a  Cl  that  includes 
a  CP/H,  was  developed  as  shown  in  Figure  1-5.  To 
support  this  time  line,  a  flow  chart  was  developed 
and  is  presented  in  Figure  1*.  5..  Finally,  a  check¬ 
list  tlat-le  i-1)  was  prepared  wh'Ch  could  be  used 
to  a;  ,ve  the  achievement  of  the  objectives  tnat 
relate  to,  'dftwtre  uUhin  a  Cl. 
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CP/H 
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BOOK  FORM  DRAWING 


1 .  DESIGN  DESCRIPTION 


CPCI  TEST  PLAN 
TEST  PROCEDURE 
TEST  REPORT 


A.  NARRATIVE 

B.  FLOWS 

C.  TRUTH  TABLES 
2..  LISTING 


Cl  ACCEPTANCE  TEST  SPEC 
ACCEPTANCE  TEST  PLAN 
ACCEPTANCE  TEST  PROCED 
ACCEPTANCE  TEST  REPORT 


OTHER 

PRINTED  CIRCUIT  ASS’Y  DWG 
SPEC  OR  SOURCE  CONT  DWG 


FIGURE  1-3:  DOCUMENTATION  FOR  SOFTWARE  WITHIN  A  CONFIGURATION  ITEM 
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PlGUU  Wi  TYTKM.  Ml  ADAOWN  Of  FUMffAJU  4  HA40WAM 
WITHIN  A  CONTICU4ATON  I  TIM 


AS 


\ 
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Reconroendatlons: 

1.  Provide  for  allowing  the  (Tyoe  A),  Sentient 
Specification  to  be  used  when  a  "software 
intensive"  CpC 1  is  embedded  within  a  Cl 
revision  to  H1L-ST0-483  is  needed. 

2.  Prepare  a  Data  Item  Description  (DID)  for  the 
Computer  Proqram/Hardwarc  (i.e.  Soot  Form 
Drawing)  and  call  for  it  on  the  CPRL .  This 
Din  should  not  require  authentication  since 

it  i  .  a  i wet  ot  the  i1r,iwtn<|  system  defined 
liy  llilU-l'-  ltlOU. 

3.  Mll-STU-130  should  be  revised  to  Include 
part  number  marling  of  firmware  media. 

4.  Develop  procedures  that  would  be  applicable 
when  both  hardware- intensive  and  software- 
mtcnsivp  microprocessor  software  reside  in 
the  sanie  Cl. 
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1A01C  1-1  CHECKLIST  I  OK  MICROl'KOCtSSOH  SOfTWAHC  JOLNTII  HATION 


Yes  No 

I.  Does  a  work  breakdown  structure/ 

specification  tree  exist  for  the  pro¬ 
gram?  _ _ 

.2.  Where  do  hardware-intensive  soft¬ 
ware  applications  exist?  Have  they 
been  identified?  _ 

3.  Are  these  applications  mutually  agree3 

to  between  contractor  and  customer?  _ 

4.  Has  a  maintenance  philosophy  at  each 
fevel  of  maintenance  been  defined  for 

these  applications?  _ 

5.  Is  the  intended  application  an  exist¬ 
ing  design?  Is  this  a  proprietary 

design?  __  __ 

6.  Have  the  firmware  developmental  re¬ 
quirements  been  adequately  defined?  _ _ 

7.  Have  configuration  management  require¬ 

ments  for  customer  design  reviews  and 
audits  been  established?  _ 

8.  Has  a  functional  baseline  been  es¬ 
tablished?  _ 

9.  Has  an  allocated  baseline  been  es¬ 
tablished?  _ _ 

10.  Has  a  preliminary  design  review  been  ” 
conducted?  _ _ 

II,  Has  tht  software  development  environ¬ 
ment  been  identified  and  documented?  _ 

(what  versions  of  assemblers,  compilers, 
linking  loaders  will  oe  employed?) 

12.  Has  an  engineering  release  schedule 
for  the  firmware  under  development 

been  prepared?  _ 

13.  Have  design  documentation  guidelines 

beer  established?  _ 

14.  Have  adequate  internal  controls  been 

established  for  version-tracking  of  _ 

the  application  progr.am(s)  under  de¬ 
velopment?  _ 

15.  What  physical  media  have  been  chosen 
for  the  firmware  in  question  (ROli, 

EPROM,  PROM,  EAROM,  etc)’  Have  docu¬ 
mentation  requirements  for  that  medium 

been  adequately  defined’  _ 

16.  Have  firmware  media  marking  require¬ 
ments  bnrn  defined? 

17.  Has  a  Critical  Design  Kevlew  been  con¬ 

ducted  on  l no  Computer  Program  Hard¬ 
ware  (CP 'M)  in  question?  _ 

13.  Have  tne  production  firmware  master 
tapes  been  released  th, ough  the  en¬ 
gineering  release  system?  _ 

19.  Have  firmware  quality  assurance  re¬ 
quirements  been  identified?  _  _ 


Yes  No 

20.  Does  the  tost  documentation  call  for 
both  read/write  and  read-only-memory 

test  phases?  _ 

21.  Is  final  validation  testinq  performed 

with  read-only-memory  as  configured  for 
delivery?  _ 

22.  Has  a  functional  Configuration  Audit/ 

Physical  Configuration  Audit  been  per¬ 
formed?  _ 

23.  Has  a  product  baseline  been  establish¬ 
ed? 


SUBPANEL  REPORTS 


0-2  Configuration  Control 


Subpanel  members  were  as  follows: 

DeBerry,  Robert  D. 

Heim,  William  J. 

Hurwitz,  Martha  H. 

Kessel,  Robert 
Martin,  Daniel 
*  McKinney,  Arthur 
Ottone,  Michael  N. 

Paslosky,  Paul 
Sabb,  Elijah 


*  Subpanel  Chair 


The  objectives  of  this  subpanel  we^e  as  follows: 

(a)  Define  Class  I/II  changes  for  nicro- 
processor  software,  usino  El A  Bulletin 
4A  as  a  guide. 

(b)  Prepare  a  flow  chart  which  depicts  the 
activities  required  to  control  micro¬ 
processor  software  changes. 

(c)  Prepare  a  checklist  that  can  be  used  to 
assure  that  microprocessor  software 
chanoes  are  adequately  controlled. 

To  satisfy  these  objectives,  the  subpanel  in¬ 
vestigated  configuration  chanoe  controls  for 
microprocessor  software  that  is  treated  as  a 

(a)  Configuration  Item  (Cl)  . 

(b)  Computer  Program  Conf i duration  Item  (CPCI), 
or 

(c)  Software  within  a  hardware  Cl  • 

Subpanel  members  concluded  that 

(a)  Configuration  control  need  only  be  applied 
to  tne  microprocessor  device  and  the  soft¬ 
ware  (code  and  data). 

(b)  Once  a  microprocessor  device  is  released 
(Enqineerinq  Release)  for  use,  changes 
should  be  processed  via  the  methodology 
utilized  for  hardware  CIs. 
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(c)  Changes  to  microprocessor  software  should 
be  processed  usino  the  existinn  methodology 
for  CPCIs. 

(d)  Formal,  internal  change  control  for  micro¬ 
processor  software  should  occur  at  the  com¬ 
pletion  of  unit  test  and  prior  to  the  start 
of  subsystem  integration  test. 

Figure  2-1  depicts  the  "Seouence  of  Events  for  A 
Generic  Project/Program" .  It  should  be  understood 
that  for  a  snecific  prcject/prooram ,  events  may 
deviate  from  those  depicted.  The  triangles  (As  ) 
are  used  tc  identify  items  released  for  use  after 
specific  events  occur.  After  release,  either  in¬ 
ternal  (company)  or  external  (customer)  change 
control  procedures  should  be  applied. 

Figure  2-2  is  a  flow  chart  that  illustrates  the 
change  control  activities  reouired  for  micropro¬ 
cessor  so'tware  after  engineering  release.  Respon¬ 
sibility  for  change  processing  ranges  from  a  single 
control  Doint  consisting  of  the  Engineering 
Manager  to  a  formal  Change  Control  Board  (CCB). 
Detailed  change  reco'nendatirns  are  submitted  to 
the  farma'  CCB  for  final  review  and  acceptance  or 
rejection.  The  "Analyze  Recuired  Change;  Assess 
Impact"  rectanole  recoanizes  a  need  to  also  re¬ 
view  baseline  documentation  for  necessary  changes. 


FIGUtt  2-2:  MKTtOTCOClSSO*  SOf  1WAM  CHANGE  CONTtOl  FLOW  CHAAT 
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TABLE  2-1  CHECKLIST  FOR  MICROPROCESSOR  SOFTWARE  CHANGE  CONTROL 


1.  Has  a  configuration  identi  ficatim 
system  been  established? 

(a)  Are  release  rijtes  available? 

(b)  Is  i  ducument/specification  t.,te 
ava  i I  able? 

2.  Has  a  problem  reporting  s.ystem  been 
defined? 

(a)  Have  forts  anj  Cortnat  been  de- 
.  fined? 

(b)  Have  change  con!-pl  approval 
authorities  been  identified? 

(c)  Has  a  problem  number/ track iny 
system  been  established? 

(H)  Has  a  mean^,  of  software  problem 
classification  been  categorized? 

3.  Has  the  iirplfmentation/verification 
procedure  been  documented? 

4.  Have  the  procedures  for  distributing 
releusc,  problem  and  change  documenta¬ 
tion  been  develcped? 

5.  Have  logistic  requirements  been  iden¬ 
tified? 


Yes  No 
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0-3:  CONFIGURATION  STATUS  ACCOUfTINfa 


Subpanel  members  were  as  follows: 

Gainey,  Ann  M. 

Hartman,  L.  Alvin 
*  Roggero,  Vincent  R. 

To.-ba,  Michael 


1  Subpanel  Chair 

The  objectives  of  this  subpar.el  were  aa  follows: 

(a)  Define  internal  and  contractual  require¬ 
ments  for  microprocessor  software  con¬ 
figuration  status  accounting. 

(b)  Prepare  a  flow  chart  depicting  the  ac¬ 
tivities  required  to  achieve  adequate 
configuration  status  accounting  for 
microprocessor  software. 

(0  Prepare  a  checklist  that  can  be  used  to 
assure  the  achievement  of  configuration 
status  accounting  objectives  for  micro¬ 
processor  software. 

The  subpanel  investigated  configuration  status 
accounting  requirements  for  microprocessor  soft¬ 
ware  that,  is  treated  as  a: 

(•)  Configuration  Item  (Cl), 

(b)  Computer  Program  Configuration  Hem  (CPCI), 
or 

(c)  Software  within  a  hardware  Cl. 

Subpanel  members  concluded  that: 

(a)  When  microprocessor  software  is  treated 
as  a  Cl.  configuration  status  accomting 
methods  used  for  hardware  are  applicable. 

(b)  Configuration  status  accounting  methods 
applicable  to  software  should  be  used 
when  microprocessor  software  is  treated 
as  a  CPC1  or  a  component  within  a  hard¬ 
ware  Cl. 


5  A 


Naval  Weapons  Slyfiort  Center 

Lccaheed 

Trident  Command 

NAVSHJR 


The  flowchart  fhb'm  1r  Ficute  3-1  re, 'loots  the 
diyision  of  activity  between  rontraclaal  e.id  in- 
itrnal  requirements  for  configuration  status 
accounting  as  di  .e.opmen.  progresses  th-o'igh 
specific  milestones.  '•  •** 

f.gure  3-?  depicts  the  flow  ov  activities  that 
are  rtqul’co  lo  achieve  aoecjat'r  configure  :ion 

;mur.  accounting  fbf  microprocessor  -ot'twar..  It 
I''  ‘mpprativ?  that  the  acciunting  tasks,  drsiq- neo 
by  triangles  (  As),  he  started  at  thj  tire  1  r-ii - 
catea.  Hence,  the  status  act;  nting  task  increases 
as  document,  hardwire  and  Suftwai  (  ■'terns,  and  in¬ 
tegration  as’-H’blios  are  added. 

k  "Tonfigurttior  Status  Ace ,'v  'ting  Checklist  far 
Micro;  rocp'.sor  Software”  is  present  )  in  Table  ?■!. 
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TABLE  3-1  CONFIGURATION  STATUS  ACCOUNTING  CHECKLIST  FOR  MICROPROCESSOR  SOFTWARE 


1. 

2. 

3. 

A. 

5. 

6l' 

7. 

8. 

9. 

10. 


Yes  No 

Have  policies  and  procedures  For  con¬ 
figuration  management  been  published?  ___  _ 
Does  a  procedure  for  internal  change 

control  exist?  _ 

Has  a  Configuration  Change  Board  (CCB) 
been  established  (single  CCB  for  hard¬ 
ware  and  microprocessor  software)?  _ 

Have  different  levels  of  change 
approval/authority  been  established  for 
different  states  of  development/produc¬ 
tion?  _ 

Does  a  numbering/identification  system 
exist  for: 

Microprocessor  Software  _ 

Related  Documentation  _ 

Does  a  system  for  approval/quality 
control,  prior  to  release/acceptance. 
exist  for: 

Microprocessor  Software?  _ _ 

Related  Documentation?  _ 

Has  a  library  control  procedure  been 
published  for  microprocessor  software 

and  documentation?  _  _ 

Has  a  configuration  management  account¬ 
ing  sysU.n  been  established  for  record¬ 
ing  and  reporting  the  status  of  proposed, 
approved,  and  implemented  changes  for 
hardware,  microprocessor  software,  and 

related  documentation?  _ 

Does  the  configuration  status  account¬ 
ing  system  provide  for  timely  and 
accurate  reporting  of  the  information 
needed  by  management  or  procuring  acti¬ 
vity?  _ 

Does  a  procedure  exist  that  provides  for 
for  an  audit/quality  control  check  of 
microprocessor  software  to  insure  that 
it  matches  configuration  documentation 
prior  to  integration  into  higher  level 
assemblies,  burn-in  of  microprocessor 
devices,  or  acceptance  by  the  procuring 
activity?  _ 
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0-4:  CONFIGURATION  AUDITS 


Subpanel  members  were  at  follows: 

*  Hawks,  Kenneth 
Hoyt.  Richard 
McAlister,  Clyde  0. 

Parks,  Frederick 
Stees,  Mae  S. 

Telxelra,  Stephen  M. 

Whxtloc*.  James  A. 


*  Subpanel. Chair 

The  objectives  of  this  subpanel  were  as  follows: 

(a)  Prepare  a  typical  Cross  Reference  Test 
Verification  Matrix  for  mi coprocessor 
software. 

(b)  Propose  method(s)  for  directing/deter¬ 
mining  the  physical  identification  of 
microprocessor  software. 

(c)  Prepare  a  flow  chart  which  depicts  the 
activities  required  to  conduct  audits 
for  microprocessor  software. 

(d)  Prepare  one  or  more  checklist  that  can 
be  used  to  -insure  that  microprocessor 
software  audits  are  conducted  properly. 

The  subpanel  investigated  configuration  audit 
requirements  for  microprocessor  software  that 
is  treated  as  a: 

(a)  Configuration  Item  (Cl), 

(b)  Computer  Program  Configuration  Item  (CPCI), 
or 

(c)  Software  within  a  hardware  Cl. 

Suhpanrl  conclusions  arc  as  follows: 

(a)  The  Verification  Cross  Reference  Matrix 
(VCRM)  shown  in  Figure  4-1  was  developed 
for  microprocessor  software.  It  is  a 
modification  of  the  M1L-STD-483  version. 
This  VCRM  is  applicable  because  micro¬ 
processor  software  dust  undergo  the 
same  tests  as  any  other  softwa.e. 
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(b)  identification  of  microprocessor  devices 
is  important;  however,  a  definitive  con¬ 
clusion  did  not  result.  Several  labeling 
methods  were  discussed,  but  each  one  had 
drawbacks.  The  methods  discussed  are 
surmarized  in  Table  4-1. 

(c)  A  flow  chart  depicting  the  activities  re¬ 
quired  to  conduct  audits  for  microprocessor 
software  was  not  preoared  for  lack  of  time. 

(d)  It  was  determined  that  a  FCA  and  preliminary 
PCA  should  be  performed  for  microprocessor 
software  prior  to  irtenration  into  the 
microprocessor  device.  The  preliminary 

PCA  would  involve  examination  of  the  micro¬ 
processor  software  on  a  sampling  basis  with¬ 
out  establishing  a  product  baseline.  Once 
system  level  testing  is  completed,  the 
microprocessor  software  PCA  and  FQR  would 
be  held  jointly  with  the  microprocessor 
device  (hardware)  PCA  and  FQR.  (See  Fig¬ 
ure  4-2). 

(e)  Microprocessor  software  eudit  checklists 
were  prepared  ior  the  FCA,  PCA,  FQR,  and 
minutes  of  these  audits/reviows,  and  are 
presentnd  in  Tables  4-?,  4-3,  4-4,  and  4-S, 
rr-.iu'Ctlvrly .  Sum'  of  the  I  Inn*.  I  Kletl 
may  not  hr  ainil  If.iblo  lor  a  given  procure- 
mrnt.  Users  should  tailor  the  list  to 
their  needs. 

Side  issues  that  were  ciscussed  durino  oanel 
deliberations  are  as  follows: 

(a)  The  difference  between  a  software  version 
and  revision. 
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(b)  Iracrabil ity  of  microprocessor  softwart* 
via  the  piun  nee  ring  release  system. 

jc)  Oedassification  of  non-eraseable  micro¬ 
processor  devices. 

Reconwendations: 

As  part  of  the  military  qualification  process. 

the  responsible  agency  should: 

(a)  Provide  potential  users  with  suggested 
method  of  identifying/labeling  micro¬ 
processor  devices  which  contain  software, 
and 

(b)  Provide  guidance  on  applicable  constraints 
with  respect  to  the  erasure  of  classified 
information  contained  within  microproces¬ 
sor  devices. 
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FULL  SCALE  ENGINEERING  DEVELOPMENT 


PRODUCTION 


fQTl 


M 


SOFTWARE^ 
HARDWARE 
INTEGRATION 
TESTING 


Cl  QUALIFI¬ 
CATION 
TESTING 


CP/H  FCA 

PRELIMINARY 

PCA 


SYSTEM 

TESTING 


PRODUCTION' 

DEPLOYMENT 


ATP 

APPLICATION 


PCA/FQR 


FIGURE  4-2«  PROPOSED  AUDIT/REV1EW  EVENTS  FOR  MICROPROCESSOR  SOFTWARE 
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Method 

1.  Stick-on  Tape 

« 

2.  Ink  Numbering 

3.  Burn  and  Bag 

4.  Color  Coding 

5.  Decals 

6.  Higher  Assembly 
labeling 


TABLE  4-1  IDENTIFICATION  METHODS  FOR  MICROPROCESSOR  DEVICES 


Remarks 

1.  Relatively  easy  to 
use. 

2.  Usually  not  harmful  . 
to  microprocessor 
device. 

3.  Dissolves  or  slides 
off  when  heated. 

1.  Easy  to  read  and  rub 
of'. 

2.  Some  Inks  produce 
static  charges. 

3.  Difficult  to  record 
numbers  in  space 
provided. 

1.  Unlabeled  micropro¬ 
cessor  device  is 
placed  in  a  container 
which  is  subsequently 
labeled. 

2.  If  item  is  separated 
from  container,  iden¬ 
tity  is  lost. 

1  Resistor  color  code 
can  be  used. 

2.  Color  code  is  confus¬ 
ing. 

3.  Government  agencies 
object  to  it. 

1.  Easy  to  apply  and  mad. 

2.  Can  produce  static 
charge. 

3.  Few  vendor  sources. 

1.  Unlabeled  microproces¬ 
sor  device  is  installed 
on  printed  circuit 
board  which  is  subse¬ 
quently  identified. 


64 


iiimiiilrinr^imrtiikiii  ir  ■ 


TABLE  4.2  FCA  CHECKLIST  FOR  MICROPROCESSOR  SOFTWARE 


1.  Review  minutes  end  corrective  ection  fro* 
previous  design  reviews. 

2.  Evaluate  Contractor's  briefing  of  functional 
requirements  and  test  results. 

3.  Audit  Preliminary  Qualification  Test  (PQT ) 
and  Formal  Qualification  Test  (FQT)  reports 
against  actual  test  data  and  procedures. 

4.  Validate  test  report(s). 

5.  Review  approved  ECPs  for  implementation 
status. 

6.  Review  interface  requirements  and  related 
test  results. 

7.  Identify  parameters  not  verified  during 
PQT  and  FQT. 

8.  Identify  support  software  used  during  de- 
velopment  and  test. 

9.  Review  deliverable  documentation  for  com¬ 
pliance  with  applicable  Data  Item  Descrip¬ 
tion  (DID). 

10.  Insure  that  pre-FCA  checklist  is  completed. 

11.  Review  failure  modes  and  effect  verification 
data. 

12.  Review  PCA  procedures  and  requirements,  if 
FCA  and  PCA  are  not  held  jointly. 
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TABLE  4-3  PCA  CHECKLIST  TOR  MICROPROCESSOR  SOFTWARE 


A 


1.  Review  minutes  and  corrective  actions  from 
previous  design  reviews  and  audits. 

2.  Compare  microprocessor  software  listing 
with  narratives  (HIPOs,  flow  charts,  etc) 
in  C5  specification. 

3.  Compare  narratives  with  requirements  in  BS 
specification. 

4.  Review  deliverable  documentation  for  com¬ 
pliance  with  applicable  Oata  Item  Descrip¬ 
tion  (DID). 

5.  Review  interface  control  documentation  for 
currency. 

6.  Review  approved  ECPs  for  Implementation 
status. 

7.  Review  applicable  deviations  and  waivers. 

8.  Review  OD2SO. 

9.  Reveiw  Acceptance  lest  Plan  (ATP)  for 
currency. 

10.  Review  nomenclature/numbering  scheme. 


TABLE  4-5  CHECKLIST  FOR  MINUTES  OF  MICROPROCESSOR  SOFTWARE  AUDITS  AND  REVIEWS 


1.  Corrective  action  completion  dates  Are  es¬ 
tablished  and  recorded  for  ell  discrepan¬ 
cies  revealed. 

2.  All  requirements  not  tested  are  recorded. 

3.  Revision  level  of  all  deliverable  documents 
are  recprded. 

4.  Unacceptable  recommendations  and  the  reason 
for  rejection  are  recorded. 
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0-5:  HORARY  CONTROLS 


Subpan# 1  members  were  as  follows: 

•  Hall,  Harold  0. 

Lett,  Alva'r. 
hegrel  1 1 ,  Thomas  J. 

Reinish,  neter 
Scnnackenberg,  Walter 


Tex.u  Instruments 
tenoral  Dynamics 
HO  AFLC/LOEE 
Hughes  Aircraft 
General  Electric 


*  Subpanel  Chair 

The  objectives  of  this  subpanel  were  as  follows: 

(a)  Review  £  I A  Bulletin  4C ,  Computer  Software 
libraries,  and  determine  its  applicability 
to  microprocessor  software. 

(b)  Prepare  a  flow  chart  which  depicts  the 
activities  required  to  implement  library 
controls  for  microprocessor  software. 

(c)  Preoare  a  checklist  that  can  be  used  to 
assure  the  adequacy  of  library  controls 
for  microprocessor  software. 

Thi-  subPunel  examined  litrary  control  require¬ 
ments  for  r- 1 coprocessor  software  that  is  treated 
as  a: 

(a)  Com -.auration  Item  (Cl), 

io)  Computer  Program  Configuration  Hem  (CPC1). 
or 

(c)  Software  within  a  hardware  Cl. 

Subpane)  members  concluded  that: 

(a)  E I A  Bulletin  4C  is  not  applicable  when 
micronrocessor  software  is  treated  as  < 

Cl.  Hardware  disciplines  would  apply. 

(b)  When  Mirroprnrossor  software  is  treated 
.is  a  i  IT  1  or  Mil  tw.no  within  .1  Cl,  CIA 
Bill  lot  in  4C  is  applicable. 

(c)  Fiqure  5-1,  Master  library  and  Programming 
Supnort  library  Phasing,  and  finure  5-Z, 
Computer  Software  Libraries  and  Rpnository 
Relationships,  depict  the  activities  re¬ 
quired  to  implement  library  control  for 
microprocessor  software.  'These  flows  were 


extracteo  from  E1A  Bulletin  4C. 


(d)  Table  5-1  presents  a  checklist  that  can 
be  used  to  assure  the  adequacy  of  micro¬ 
processor  software  library  controls. 
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TABU  5-1  CHECKLIST  FOR  MICROPROCESSOR  SOFTWARE  LIBRARY 


Does  the  microprocessor  software  library  contain: 

Yes  No 

(a)  Source  Code,  both  for  application 

software  and,  as  applicable,  sup¬ 
port  software?  _ 

(b)  Object  Code,  both  for  application 

software  and  support  software?  _ 

(c)  Load  module?  _ 

(d)  Source  listing,  both  for  applica¬ 
tion  software  and,  as  applicable, 

support  software?  _ 

(e)  Onject  listing,  both  for  applica¬ 
tion  software  and  support  software?  _ 

(O  Test  plan,  test  procedures,  test 
case  source  media  and  test  reports 
associated  with  the  qualification 
of  the  application  software?  _  _ 

(g)  Software.  Standards  and  Procedures 

Manual?  _ _  _____ 

(h)  User  manuals  for  both  target  and 
host  computers  (if  not  contained 

in  another  corporate  repository)? _ __ 

(i)  Configuration  identification 

(specifications  and  drawings)  if 
not  contained  in  another  corporate 
repository?  _ _ 

(j)  Version  Description  Document  or 
equivalent  (identifies  all  items 
which  make  ud  the  version  plus  all 
utility  and/or  support  Items/mod¬ 
ification  levels  which  are  required 
to  operate,  load  or  regenerate  the 
application  software)? 

(k)  Identification  of  equipments/mod- 
ification  levels,  including  both 
target  and  host  computers,  which 
were  used  for  testing  of  the  ver¬ 
sion  (if  not  contained  in  test 
documentation  or  VDD  as  discussed 

in  item  j  above)?  _ 

(l)  Release  Procedure? 

(m)  Oiiiiiqr  Control  Procedure? 

(n)  r  iiul  Release  Procedure?  _ 

(o)  Backup  File  Procedure?  _  ^ _ 
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PANEL  RCCOWIENDATIONS 


* 


For  Industry 

(a)  Develop  procedures  that  would  be 
applicable  when  both  hardware-intensive 
(Firmware)  and  software- intensive 
(microprocessor  software)  reside  in  the 
same  Cl. 

(b)  Develop  guidelines  for  the  erasure  of 
microprocessor  devices  containing  clas¬ 
sified  software. 

For  Industry  Associations 

(a)  Prepare  a  Data  Item  Description  (DID) 
for  the  computer  program/hardware 
(i.e.,  book- form  drawing ,  that  can  be 
called  for  as  a  Contract  Data  Require¬ 
ments  list  (CDRL )  item).  This  DID 
should  not  require  authentication 
since  it  is  a  part  of  the  drawing  sys¬ 
tem  defined  by  DoD-D-1000. 

For  Government 

(a)  Revise  Mli-STD-483  to  allow  the  use  of 
a  Type  “A"  specification  when  software¬ 
intensive  CPC  1 s  are  embedded  within  a 
hardware  Cl. 

(b)  Revise  MIL- STD- 1 30  to  include  part  num¬ 
ber  marking  requirements  for  microproces¬ 
sor  devices  containing  software. 
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APPENDIX  C-4 


DYNAMICS  RESEARCH  CORPORATION 
WILMINGTON.  MASSACHUSETTS  01887 


OUTLINE 


PANEL  B:  ISSUES 


PANEL  B:  OBJECTIVES 


PANEL  B:  REFERENCE  MATERIAL 
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THE  ENTIRE  SYSTEM  (HARDWARE  PLUS  FIRMWARE)  IS  MAINTAINED  BY  THE 
GOVERNMENT.  FULL  CSCI  DOCUMENTATION  IS  PROVIDED  BY  THE  CONTRACTOR. 


PANEL  B:.JLC  POLICY 

A 


PANEL  B:  SOME  CRITERIA 


USE 


APPENDIX  C-5 


PFOPOSED 

EIA 


CP/H  BOOK  FORM  DRAWING 


SOFTWARE  INTENSIVE 
CPCI 


HARDWARE  INTENSIVE 
CP/H 


65  SPEC 
C5  SPEC 


I 

I 


MIL-STD-490 


CPCI  TEST  PLAN 
TEST  PROCEDURE 
TEST  REPORT 


DOD-D-IOOO 


BOOK  FORM  DRAWING 

I.  DESIGN  DESCRIPTION 
A.  NARRATIVE 
6.  FLOWS 
C.  TRUTH  TABLES 
2..  LISTING 


Cl  ACCEPTANCE  TEST  SPEC 
ACCEPTANCE  TEST  PLAN 
ACCEPTANCE  TEST  PROCED 
ACCEPTANCE  TEST  REPORT 


OTHER 

PRINTED  CIRCUIT  ASS'Y  DWG 
SPEC  OR  SOURCE  CONT  DWG 


FIGURE  1-3:  DOCUMENTATION  FOR  SOFTWARE  WITHIN  A  CONFIGURATION  ITEM 
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DATA  ITEM  DESCRIPTION 

j  idcm'i,  ,c»  now  j 

AOCNcr 

NUtfar  A 

i  tifkl 

Firmware  Development  Plan  (FDP) 

USAF 

UDI-E-3935 

ASD/M2 

Th«  Firmware  Development  Plan  (FDP)  1*  that  engineering 

«  *»»*OV  AL  OA  T  | 

14  September  197') 

management  plan  which  Identifies  the  contractor's  efforts 
required  to  develop  and  deliver  computer  resources  and  the 
associated  firmware,  processes,  documentation,  and  necessary 

«  o*»iCC  or  m  m  ■  u  a  *  V 

MlPONMi 

afsc/asd/enetc 

supporc  resources.  The  plan  is  used  by  the  contracting 
activity  to  monitor  and  evaluate  the  contractor's  firmware 
computer  resource  development  effort  and  to  plan  the  support 

ft.  DOC  MQuiMKD 

of  these  resources. 

ft 

LIMIT  anon 

1.  AP  P lie  A  T  IO  HJ  iH  T  c AACi.  AT  IOHDKI* 

This  Data  Item  Description  applies  to  system  development 
and  acquisition  contracts  involving  computer  resources 
during  che  phases  of  validation,  full  scale  development 
and  production.  This  plan  is  related  to  the  Computer 

Program  Development  Plan  (CPDP),  configuration  management 
plan  and  test  plan.  The  FDP  and  CPDP  together  cover  the 
full  spectrum  of  computer  resources  being  acquired. 
Furthermore,  this  plan  is  closely  related  to  contractual 
requirements  covering  che  System  Engineering  management 
requirements  and  documentation.  UDI-E-3936-ASD  and 
UDI-E-3937-ASD  are  companion  data  Items. 

For  use  at  ASD/ENETC 
and  ASD/W 

•  Rif  (K(nc  K>  (,)ftinAei(M>  •  e  cited  in 
AJ#c ft  2  0; 

AFR  800- U 

MIL-STD-1521 

MIL-D-83468 

DI-S-30567A 

UDI-E-3936-ASD 

UDI-E-3937-ASD 

. 

ftftCDL  hOMDttlB 

>*.  AAA  T  ION  INlTMUCtlOMI 

10. 1. a  The  FDP  is  a  documented  engineering  management  plan  that  shall  define  the 
contractors  efforts  to  develop,  document,  and  deliver  computer  resources  in 
acrordance  with  the  contract  terms.  As  a  management  plan,  the  FDP  shall  analyze  and 
structure  the  work  into  manageable  elements  (components,  modules,  tasks,  sub-systems, 
etc.),  schedule  the  progress  of  these  elements,  define  milestones  and  roeasurcable 
products  for  each  milestone  or  activity  during  the  development  process  through 
delivery  of  the  product  (system).  The  FDP  shall  identify  che  organizational  structure 
to  perform  the  tasks,  the  interface  with  ocher  groups,  and  the  resources  allocated 
to  the  work  effort.  Thus,  the  FDP  shall  provide  effective  management  visibility, 
monitoring  and  control  of  che  product  development,  test,  integration,  acceptance  and 
delivery. 

lO.l.b  The  FDP  shall  be  coordinated  with  the  CPDP  (DI-S-30567A)  in  terms  of 
resources  and  schedule.  This  FDP  shall  apply  to  computer  programs,  computer  data, 
microprocessor,  digital  electronic  processor,  and  firmware  developments  and  appli¬ 
cations  or  Implementations  defined  herein  aa  Hardware  Intensive.  The  balance  of  , 

these  applications  and  developments  defined  aa  Software  Intensive  shall  be  addressed 
in  che  CPDP. 

10.2  As  a  minimum,  the  plan  shall  Include  the  following: 

.«»  ,  ' 

n  r>  ro»»  4  c  /?  4  ' 

r  •  '*  o  _•  v  <-•  +>.  «  •  •  •  »■  **'.*..  *  *  ■  **'  -  *' 


lfl>I-E-3<m-ASD/H2 


10. 2.  a  The  contractor  shall  provlda  definition  of  the  microprocessor,  Digital 
Electronic  Processor  (DEP)  snd  firmware  Intended  to  be  developed  and  covered 
In  this  FDP  ithat  is  hardware  intensive  applications)  as  a  supplement  to  the 
definitions  provided  herein.  These  definitions  shall  be  used  in  identifying 
the  applications  covered  in  this  FDP  (as  required  by  section  10. 2. b). 

10. 2.  b  Firmware  Identification.  All  planned/knovn  application  of  microprocessors, 
DEP,  and  firmware  elements  in  the  system  shall  be  identified  and  described  as  to 
functions  performed  and  implementation  technology.  An  overall  block  diagram 

and  table  shall  be  orovided  which  identifies  all  microprocessor,  DEP,  and  firm¬ 
ware  Implementations  in  the  system. 

10. 2.  c  Firmware  Development  Process.  The  FDP  shall  include  a  complete  definition 
and  description  of  the  step-by-step  process  of  implementecions  or  applying  micro¬ 
processors,  DEPs,  and  firmware  in  the  development  of  this  system  development.  A 
diagram  of  phased  deve lopment/process  steps  shall  be  included.  Each  development/ 
process  step  shall  be  completely  described  including  the  input  baseline  from  the 
previous  development/process  step,  the  output  to  be  generated  by  the  activity, 
and  the  exact  nature  of  the  activity  (see'Flgure  1).  Identify  specific  mile- 
atohes,  integral  to  this  process,  which  you  will  use  to  manage  and  crack  the 
achedule  (required  by  10, 2. e).  Where  appropriate  the  milestones  identified  in 
the  CPDP  (D1-S-30567A,  Sec  10. A. a)  may  be  used  herein.  Identify  your  method  of 
integrating  and  verifying  the  status  of  these  milestone  products  into  your 
management  control  system  and  cost  performance. 

10. 2.  d  The  structure  of  the  organisation  responsible  to  develop  the  Hardware 
Intensive  application  shall  be  provided.  This  manning  structure  shall  Identify 
ail  personnel  responsible  for  and  contributing  to  the  de f i>;i tion/deve lopment 

of  the  applications.  The  structure,  authority-,  responsibilities,  interfaces 
and  lines  of  communications  for  all  participants  shall  be  indicated.  Each 
element  of  Hardware  Intensive  application  should  be  size  estimated  in  terms  of 
instructions  and/or  data.  The  total  manpower  allocated  to  each  element  of  the 
firmware  shall  be  identified  in  terms  of  man  months  per  month  and  by  skill  levels 
such  as  analyst,  engineer,  programmer,  etc.  Recognising  this  development  effort 
to  be  closely  integrated  with  the  total  system  development  effort,  the  inter¬ 
relationship  and  interfaces  between  the  Hardware  Intensive  application  development 
organization  and  the  other  development  organizations  shall  be  defined. 

10. 2.  e  The  plan  shall  contain  a  schedule  which  defines  the  total  development 
effort.  This  schedule  shall  identify  the  development  tasks,  milestones,  and 
vrltten  documentation  product  associated  with  each  implements t ion/app licatlon 
of  microprocessors,  DEP  and  firmware.  These  tasks  shall  include: 

(1)  Definition  of  the  functional  performance  requirements  to  be 
Implemented  (allocation). 

(2)  Verification  of  allocated  functional  performance  requirements. 

(3) .  Algorithms,  equations,  and  data  structures  development  and  their 
derivations. 

(4)  Verification  of  the  derived  algorithms,  equations,  and  data 
against  the  functional  performance  requirements. 

•*  , 
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(5)  Derivation  of  the  design  through  detail  flow  process,  specific 
date  content,  end  date  addresses  (Implementation  of  the  design). 

(6)  Definition  of  Teit  criteria  and  test  procedure*  to  verify  correct 
performance  once  Implemented. 

(7)  The  process  the  coding  of  digital  data  (manual  or  otherwise  conver¬ 
sion  of  the  data,  i.e.,  micro  Instructions  into  bit  patterns  and  the 
allocation  of  the  data  to  firmware). 

(8)  Implementation  of  the  writing  of  information,  the  code  and/or 
data  into  the  firmware  device. 

(9)  Verification  that  the  programmed  device  correctly  performs  the 
Intended  function. 

I 

(10)  Integration  (incremental  build-up)  of  the  firmware  elements  into 

the  subsystems  and  system  (including  measurable  criteria  for  determining 
completion) .  ' 

The  scheduled  tasks  shall  include  the  manhours  planned  to  do  the  task 
and  the  specific  planned  personnel  by  skill  level  assignments.  These  tasks 
shall  be  scheduled  and  cracked  with  start  and  completion  milestone  dates. 

It  is  essential  that  these  milestones  be  finite  and  measurable  (written 
product).  This  schedule  shall  be  consistent  with  the  cost/schedule/reportlng 
information  required  t>y  the  contract.  Information  identified  for  each  item 
in  this  schedule  shall  Include  scheduled  completion  of  the  documentation 
milestones  (e.g.  detail  design  solutions  must  be  documented  at  the  Critical 
Design  Review  milestone).  The  contractor's  approach  to  track  and  report 
progress  relative^  to  this  devalopment  schedule  shall  he  identified. 

10.2.  f  The  planning  and  procedures  for  firmware  cosc/schedule  reporting 
shall  be  Identified.  Specific  methods,  reporting  formats,  lowest  level 
planning  status  reporting  cools  shall  form  the  baseline  for  cost  performance 
reporting. 

10. 2.  g  Planning  for  testing  shall  be  identified  to  Include  specific  plans 
for  all  Hardware  Intensive  applications.  Test  plans  shall  cover  both  con¬ 
tractor  verification  methods  and  scheduled  activities  and  Air  Force  verifica¬ 
tion.  These  plans  should  be  Integrated  with  the  simulator  system  level  test 
plans,  but  must  cover  the  Hardware  Intensive  applications  performance. 

10. 2.  h  Methods  and  procedures  planned  to  develop  firmware  product  speci¬ 
fication  information  to  meec  Che  requirements  identified  in  the  Contract 
Data  Requirements  List  shell  be  provided.  The  successive  milestone  build¬ 
up  of  the  firmware  development  documentation  in  terms  of  design  solutions 
Is  critical  and  must  be  planned  and  achieved  In  order  to  meet  the  scheduled 
design  reviews.  Detail  design  solutions,  flow  charts,  process  flows,  or 
equivalent  information  processing  designs  must  be  planned  to  be  complete  for 
Critical  Design  Review. 
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10.2.1  Method*  end  procedure*  to  accomplish  verification  of  each  develop¬ 
ment  *tep  prior  to  preceedlng  to  the  next  development  *tep  shall  be  Identified. 
For  example,  the  flow  process  or  equivalent  design  must  be  verified  against 
the  algorithm  and/or  Part  1  specification  prior  to  coding.  Techniques  used 
for  design  control  to  assure  completeness,  validity,  traceability  to  require¬ 
ment,  testability,  modularity  and  compliance  with  Internal  standards  shall 
be  Identified. 

10.2. J  Resources,  including  computer  programs,  equipment,  documentation, 
and  personnel  skills  required  to  support  the  development,  modification,  and 
test  of  the  Hardware  Intensive  applications  shall  be  Identified  and  scheduled. 

10. 2.  k  Planning  which  describes  methods  to  develop  and  control  digital  data 
base  information  shall  be  identified.  This  planning  shall  Include  methods 
to  baseline  and  control  source  and  operational  loads  and  to  document  these 
data  bases.  Levels  of  baseline  control  used  throughout  the  development 
shall  be  identified. 

t 

10.2.1  A  description  of  engineering  practices,  standards,  conventions,  design 
reviews,  etc.,  as  they  apply  to  the  development  and  how  these  standards  will 
be  maintained  shall  be  provided.  In  addition,  the  methods  to  be  employed 
to  assure  adherence  to  these  disciplines  shall  be  described. 

10.2.3  Definitional 

Firmware:  Solid  state  electronic  devices,  t.e.,  ROM,  PROM,  EPROM,  etc., 
which  contain  binary  bit  patterns  (digital  information)  that 
cannot  be  readily  modifiable,  i.e.,  read  only. 

Digital  Electronic  Processor:  An  electronic  device  whose  functions  cr 
characteristics  are  programmable  at  multiple  levels,  l.e., 
programmable  at  the  prlmatlve  microinstruction  or  h  igher 
level  (machine  instructions).  These  microinstructions  com¬ 
bine  to  make  the  microprograms  which  define  the  instructions 
set  (macroinstructions  or  machine  Instructions)  at  che  user 
level. 

Embedded:  (1)  physically  incorporated  Into  a  larger  system  or  subsystem 
l.e..  Integral. 

(2)  the  larger  system  function  Is  not  general  purpose  data 
processing. 

(3)  the  application  may  be  hardware  Intensive  or  software 
Intensive. 

HV  Intensive:  Those  microprocessor/digital  electronic  processor 

applications  in  which  the  function  performed  is  fixed  and 
for  any  change  to  occur  a  redevelopment  of  the  application 
•>»  function  would  be  required.  -  •  • 
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ISV  Intensive:  Those  microprocessor/dlgltel  electronic  processor 

eppllcetlons  in  which  che  function  can  vary  or  Is  capable 
of  performing  multiple  functions  within  the  embedded  appli¬ 
cation  (real-time  function,  calibration  function,  built- 
in  test,  diagnostic). 

Microprocessor :  An. integrated  circuit  chip  or  chip  set  which  has  a 
fixed  instruction  set  at  the  micro  or  machine  Instruction 
level  and  which  has  a  fixed  architecture. 

Programmable  microprocessor:  A  microprocessor  where  the  instruction 
,  set  Is  programmable  (modifiable)  at  the  microinstruction 

level  and  whose  architecture  is  therefore  defined  by  those 
microinstructions. 

Microinstruction:  A  bit  pattern  stored  in  high  speed  memory  (normally 
nonvolatile)  which  controls  the  processor  hardware  logic. 

e 

*'  Machine  instruction:  A  bit  pattern  that  Is  interpreted  by  the  executing 

control  hardware  of  a  processor  and  Is  made  up  of  a  sequence 
of  microinstructions. 

Macrolnstruccion:  A  mnemonic  instruction  which  causes  an  HOL  computer 
program  to  generate  one  or  more  machine  Instructions. 

ROLt  Higher  Order  Language. 
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Firmware  Technical  Description  (Product  Specifica¬ 
tion) 


Provides  complete  and  detailed  technical  descrip¬ 
tion  of  each  functional  implementation  of  firmware. 
The  document(s)  serve  as  an  instrument  for- accept¬ 
ance,  modification,  trouble  diagnosis,  maintenance, 
modification  and  reprocurement  of  the  firmware. 
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This  description  serves  to  fully  document  all 
available  information  for  each  functional  imple¬ 
mentation  of  firmware  to  be  delivered.  Tt  is 
developed  simultaneously  with  the  firmware  and  is 
used  incrementally  to  measure  development  process 
at  the  various  milestones.  It  includes  flow  charts 
and  detailed  descriptive  text  needed  "by  the  system 
programmer  and  data  analyst  in  support  of  the 
delivered  firmware.  DI-E-3120,  UDI-E-3935-ASD  and 
UDI-E-3937-ASD  are  companion  Data  Items. 
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It  INITMUtTIOM 

10.1  Prepare  a  complete  comprehensive  technical  description  of  the  firm 
ware  to  be  delivered  under  this  contract. 

Vi 

Definitions:  Firmware,  most  commonly  defined  as  computer  programs  and 
computer  data  at  the  microprogram  level,  also  applies  to  any  level  of 
executable  computer  programs  and  computer  data  that  cannot  be  readily 
modified  under  program  control,  that  is  read  only.  For  the  purpose  of 
this  data  item  description,  firmware  is  defined  to  include  the  above 
definitions  and,  in  a- broader  sense,  will  include  all  information  pro¬ 
cessing  implementation  technologies,  programs,  digital  data,  and  de¬ 
vices  not  included  under  the  definition  of  digital  computers  and  associ¬ 
ated  computer  programs,  and  not  included  under  hardware.  Firmware  in¬ 
cludes  microprocessors,  Read  Only  Memories  (ROMS),  Programmable  Read 
Only  Memories  (PROMS),  and  any  other  programmable  logic  elements.  The 
contractor  shall  expand  from  these  generic  definitions  to  specific  de¬ 
finitions  of  the  firmware  intended  to  be  covered  by  the  documentation 
technical  descriptions  in  response  to  this  data  item  description. 

10.2  Each  firmware  implementation  (e.g.  individual  or  grouped  PROM(s), 

ROM ( s ) )  shall  be  identified  and  entitled  separately  by  function  per¬ 
formed  and  .described  in  relation  to  the  associated  configuration  item 
of  which  it  is  a  part.  The  technical  data  shall  be  comprised  of  the 
followings  v  -•< 
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a.  Identif ication  of  '■he  performance  requirements  and  functions 
to  be  implemented.  This  shall  include  test  criteria  and  re¬ 
quirements  . 

b.  A  narrative  description  of  the  functions  to  be  per¬ 
formed,  the  inputs  and  outputs  associated  with  the  functions, 
and  the  hardware  and  software  interfaces. 

c.  A  description  of  the  functional  characteristic  of  t-he 
devices  to  be  programmed  and  the  types. of  information  struc¬ 
turing,  e.g.  microinstruction/data  formats  nnd  instruction 
type  functions.  As  appropriate,  device  micro  instructions 
shall  be  defined. 

d.  A  complete  description  of  the  firmware  device (s)  to 
include  memory  size  (length,  width  in  bits),  operating 
characteristics  (access  time,  power  supply/requirements,  logic 
levels,  etc.),  pin  functional  descriptions  and  logical  inter¬ 
faces,  and  manufacturers  part  number.. 

e.  A  complete  definition  of  the  algorithms,  equations 
and  data  structures  to  be  implemented.  Flow  processes  and 
data  formats  of  implementation  shall  be  described  at  the  de¬ 
tailed  functional  level. 

f.  Logic  layout  schematic  type  block  diagrams.  These 
shall  include  timing,  cycle,  and  clock  information. 

g.  Detail  process  flow  diagrams  which  support  coding  as 
the  next  step.  Data  structures,  e.g.  look  up  tables,  shall 
be  completely  detailed  including  addresses  and  hex/decimal 
equivalents.  The  derivation  of  table  entries  shall  be  pro¬ 
vided  including  descriptions  of  the  equation/algorithms  which 
generated  the  data. 

h.  The  source  code  of  instructions  and  data.  Listings 
of  the  firmware  contents,  sequentially  by  address,  in  binary, 
hex  or  decimal  and  mnemonic  representation  or  descriptions 
shall  be  provided.  Listings  shix’  contain  comments  which 
accurately  correlate  to  the  flow  process  diagrams  used  to 
generate  the  contents.  These  listings  may  take  the  form  of 
parenthetical  drawings  of  the  contents  of  the  firmware. 

i.  A  listing  of  the  firmware  contents  (i.e.,  the  logic/ 
program  contained  in  memory)  in  either  a  binary  or  mnemonic 
representation  referenced  to  a  logic  diagram  graphical 
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FIRMWARE  SUPPORT  DATA 


data  item  oucripticm 


'OCNllflC  ATION  atom 


Firmware  Support  Data 


Provides  the  data  needed  for  the  maintenance  and 
modification  of  firmware  (Read  Only  Memories  (ROMs) 
Programmable  Read  Only  Memories  (PROMs) )  micropro¬ 
cessors,  and  other  firmware  devices. 
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This  data  serves  to  fully  document  all  available 
information  for  the  maintenance  and  modification  of 
firmware.  It  is  to  be  used  as:  (1)  a  maintenance  ‘ 
manual  by  the  system  programmer  in  support  of  the 
delivered  firmware,  and,  (2)  engineering  documenta¬ 
tion  for  depot  level  support.  DI-E*-3120,  UDI-E- 
3935-ASD  and  UDI-E-3936-ASD  are  companion  Data  Item;. 
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and  ASD/ENETC 


».  >[rtR(NCIi  i 

tO) 


AFR  000-1“ 
MIL-STD-1 521 
MIL-D-83468 


mcil  >uuil>ii 


>«.  INITflUCTlOHI 

Unless  otherwise  stated,  the  following  information  will  be  provided  sepa¬ 
rately  for  each  functional  implementation  (i.e.,  individual  or  grouped 
ROMs/pROMs  and  other  devices)  of  firmware. 

a.  The  object  rode  for  each  ROM,  PROM  or  other  device  (as  input  to  a 
PROM  (device)  programmer)  on  punched  paper  tape,  magnetic  tape  or  other 
specified  medium  supplemented  with  a  listing  of  the  object  code  and  a 
narrative  descript.i  on  of  the  complete  contents  -~>f  the  tape  or  other  speci 
fied  medium  (to  include  the  leader,  etc.).  If  the  listing  is  on  perfora¬ 
ted  forms,  the  pages  shall  not  be  separated.  The  listing  shall  contain 
both  address  and  content  (in  binary,  octal  or  hexadecimal)  to  each  indi¬ 
vidual  (for  PROMs  and  other  programmable  devices)  ROM,  PROM  or  other  de¬ 
vice.  A  listing  of  source  language  statements/code  used  at  each  stage  of 
development  shall  be  provided.  All  listings  shall  be  annotated  to  ident: 
fy  computer  program  identification  member  (CPIN) ,  version,  date  of  listin 
system  memory  location,  card  memory  location,  and  chip  memory  location. 

b.  A  Programming  Guide  (for  PROMs  and  other  programmable  devices) 
ROM,  PROM  or  other  device  shall  be  provided  in  contractor  format.  This 
guide  shall  identify  all  hardware/software  and  describe  the  procedures, 
used  for  maintenance  and  support  of  both  the  PROMs  (devices)  and  the  in¬ 
formation  content  of  the  PROMs  (devices).  The  content  of  this  guide  shai 
include: 


_ ^  list  of  all  equipments  for  each  functional  implementation 
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of  firmware  including  computers  and  computer  peripherals  used  by  the 
contractor  for  software  developnent/data  base  generation, 

PROM  loading,  PROM  burn-in  and  PROM  test  (including  verifi¬ 
cation  that  the  proper  content  is  stored)  ..  Each  equipment 
shall  be  identified  by  manufacturer,  manufacturer's  designa¬ 
tion,  and  any  special  features.  Any  unique  equipments  used 
shall  be  so  specified. 

(2)  A  list  of  all  computer  software  for  each  func¬ 
tional  implementation  of  firmware  used  by  the  contractor  for 
firmware  logic  development/data  base  generation,  device,  e.g., 

PROM  loading,  burn-in  and  test.  All  cross-compilers,  cross 
assemblers  and  utility  programs  used  will  be  included  in  this 
list.  Each  computer  software  item  shall  be  identified  by 
vendor,  vendor's  designation,  version,  and  any  special  fea¬ 
tures.  Any  unique  software  used  shall  be  so  identified. 

(3)  A  sequential  list  for  each  functional  implemen-  • 
tation  of  firmware  of  those  detailed  procedures  used  by  the 
contractor  for  firmware  logic  development/data  base  genera¬ 
tion,  device,  e.g.,  description  of  PROM  loading,  burn-in,  and 
test.  All  equipment  and  computer  software  used  in  each  pro¬ 
cedures  shall  be  identified.  All  burn-in  schedules  and  con¬ 
ditions  used  by  the  contractor  shall  be  included. 

(4)  a  list  for  each  functional  implementation  of 
firmware  of  those  tests  used  by  the  contractor  to  validate 
firmware  logic/data  base,  compatibility  of  the  firmware  logic/ 
data  base  with  the  system,  device  (e.g.  ,  PROM)  specification 
and  device  (e.g.,  PROM)  content  after  programming.  A  brief 
overview  of  each  test  will  be  included.  The  test  method  and 
criteria  for  acceptance  or  rejection  will  be  included  for 
each  test.  Those  equipments  and  computer  software  used  for 
each  test  will  be  identified. 

c.  Vendor/Ireplemen tation  Information. 

(1)  Vendor  data  as  supplied  by  original  supplier 
and/or  vendor  which,  as  a  minimum  describes  in  detail,  the 
capabilities  and  methods  of  achieving  these  capabilities  for 
each  device. 

(2)  Pin  layout  and  logic  diagram  relation  to  each 
pin  shall  be  provided. 
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CONFIGURATION  ITEM  PRODUCT  FABRICATION  SPECIFICATION 


OATA  ITEM  DESCRIPTION 


Configuration  Item  Product  Fabrication  Specification 


[».  oci<«i*»iON/m«*on 

The  product  configuration  identification  documentation 
(specifications  and  referenced  drawings)  establish  the  re¬ 
quirements  for  manufacture  and  acceptance  of  the  configura 
cion  items  (Cl)  to  be  delivered  under  the  terms  of  the 
contract . 
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MIL-STD-483 

MIL-STD-490 

MIL-S-83490 


’ »t,o«»mi*  ihe  product  labrication  specttlda- 
tion  is  used  to  identify  tne  contractual  requiranents  (proauc  : 

baseline)  for  the  configuration  item.  The  product  specif icati >n 
is  the  documentation  to  which  production/operational  engi¬ 
neering  change  proposals  (ECPs)  are  addressed.  The  product 
fabrication  specification  will  normally  be  prepared  as  Part 
11  of  a  two  part  specification.  Part  I  of  the  specification 
will  always  be  the  overriding  part  of  the  specification.  For 
Computer  Program  Configuration  Items  (CPCIs),  use  DID-E-312GB 
If  other  than  Form  la  s pecificat ions (MIL-STD-490)  are  requir¬ 
ed,  the  specific  form  from  MIL-S-83490  will  be  called  out  in 
block  16  of  the  Contract  Data  Requirements  List  (CDRL) .  For 
configuration  items  that  contain  embedded  microprocessor  ap¬ 
plications,  whose  computer  programs  are  not  separate  computer 
program  conf igur aiton  items,  the  additional  requirements  con¬ 
tained  below  in  Bock  10  (para. 2),  Preparation  Instructions, 
shall  be  included  in  the  specification.  For  Prime  Items  that  ^ 
are  training  eouipments,  the  additional  requirements  contain¬ 
ed  below  in  Block  10  (para  3).  Preparation  Instructions,  shalL 
be  included  in  the  specification. 

10  »«»  T  ION  INITHUC  TION1  . 

1.  The  contractor  shall  prepare  a  product  fabrication  specification  for  each  configur¬ 
ation  item  in  accordance  with  the  requirements  of  MIL-STD-490,  Appendices  VIII,  X  and 
XI,  as  applicable  and  as  stated  in  the  contract  or  work  statement.  When  other  than  Form 
la  specifications  are  called  out  in  Block  16  of  the  CDRL,  Appendices  of  MIL-STD-490  wil 
be  used  as  a  guide  in  the  preparation  of  specifications.  The  specifications  cover  page 

shall  be  in  accordance  with  MIL-STD-483,  Figure  1. 

• 

2.  When  the  configuration  item  contains  embedded  microprocessor  applications,  an  ap¬ 
pendix  for  each  microprocessor  application  shall  be  included  in  the  conf iguralton  item 
product  fabrlacation  specification.  This  appendix  shall  contain  additional  information 
about  the  microprocessor  application.  The  section  in  the  product  specification  which 
contains  the  chip  schematics  and  bit  pattern  drawings  shall  reference  this  appendix. 
Title  and  numbering  of  the  appendix  shall  be  in  accordance  with  MIL-STD-490.  Detailed 
preparation  instructions  shall  be  implemented  by  the  following  instructions. 

Vicrlon  1,  Scope:  This  section  shall  identify  the  microprocessor  and  describe  its 
function  within  the  configuration  item. 

Section  2,  Applicable  Documents:  References  which  may  be  required  and  which  relate 
to  this  appendix  shall  be  listed  iv.  Section  2  of  the  basic  specification  and  shall  be 
listed  by  the  title  in  this  section  as  e  minimum.  If  there  are  no  applicable  reference 
the  following  shall  appear  below  the  section  heading: 

"This  section  is  net  applicable  co  this  appaidix." 
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Section  3,  Requirements!  All  of  the  requirements  which  are  allocated  to  the  micro¬ 
processor  application  iiTtjTi  be  clearly  stated,  either  directly  or  by  reference,  in  thi* 
section.  Any  requirements  derived  as  a  consequence  of  the  application  of  a  mlcroprocc' 
In  the  configuration  Item  shall  be  included  in  this  section.  Derived  requirements  may 
be  allocated  to  t ho  digital  processing  equipment,  the  microcode,  computer  program  or 
data,  and  may  include  processing  rates,  memory  usage,  programming  language (s ) ,  data 
structures,  etc. 

Paragraph  3,1,  Instruction  Set:  Identification  of  the  programming  language  and 
instruction  set  shall  be  included  either  directly  or  by  reference  in  this  paragraph. 

If  t he  microprocessor  is  a  user  programmable  microprocessor  and  an  instruction  set  was 
developed,  a  complete  description  of  what  happens  with  the  execution  of  each  instructic 
shall  be  included  either  directly  or  by  reference,  along  with  a  memory  map  for  the 
instruction  set  In  this  paragraph  and  subparagraphs,  If  the  microprocessor  is  a  user 
programnab le  microprocessor,  but  an  instruction  set  was  not  developed  (i.e.,  the  micro¬ 
processor  is  used  as  a  hardware  controller),  then  a  complete  description  of  the  control 
flow  sequencing  and  timing  shall  be  included  in  this  paragraph  and  subparagraphs .  If 
the  microprocessor  is  user  programmable ,  but  an  instruction  set  was  not  developed, 
paragrpalis  3.2  through  3.5  of  this  appendix  will  not  be  applicable. 

Paragraph  3.2,  Functional  Flow:  This  paragraph  shall  describe  the  major  operation(t 
performed  by  the  computer  program.  This  paragraph  and  subsequent  subparagraphs  shall 
show  t he  general  flow  of  both  data  and  control  within  the  computer  program.  Paragraph 
3.2  shall  graphically  portray  the  operations  performed  by  the  computer  program. 
Graphical  representations  shall  be  in  accordance  with  MIL-STD-483,  Appendix  VI. 

Paragraph  3.2.1,  Interfaces:  This  paragraph  shall  describe  in  detail,  cither 
directly  or  by  reference,  the  as-built  interface  design  between  the  computer  program 
and  the  hardware  with  which  it  must  operate.  It  shall  provide  a  detailed  logical 
description  of  all  data,  messages,  and  control  signals.  Sources  for  all  Inputs  and 
destinations  for  all  outputs  shall  be  described. 

Paragraph  3.2.2,  Program  Interrupts:  This  paragrpah  shall  describe  all  program 
interrupts.  Each  interrupt  shall  be  described  as  to  source,  purpose,  type,  priority, 
and  the  required  response.  The  probable  rate  of  occurrence  of  interrupts  shall  also 
be  given.  If  the  computer  program  is  loop  driven,  then  loop  rates  (inner  loop  rate(s), 
outer  loop  rate(s),  etc.)  shall  be  described.  The  effect  of  interrupts  on  those  loop 
rates  shall  also  be  described. 

Paragraph  3.3,  Design  Description:  The  design  description  provided  shall  be  to  a 
level  of  detail  sufficient  to  permit  computer  program  modification  and/or  adaptation 
by  the  government.  For  applications  specified  by  the  government  as  Hardware  Intensive, 
the  level  of  detail  shall  be  sufficient  to  provide  for  the  theory  of  operation  of  the 
Configuration  Iteln  and  to  facilitate  applicable  maintenance  at  the  hardware  level. 

Paragraph  3.3.1,  Data  Base  Definition:  This  paragraph  shall  include  the  name, 
definition,  variable  type,  and  rapge  of  values  for  each  variable,  table,  or  array 
>  used  in  the  computer  program. 

Paragraph  3.3.2,  Storage  Allocation:  This  paragraph  shall  include  a  memory  map 
which  describes  the  utilization  of  all  portions  of  memory.  This  memory  map  shall 
include  a  description  of  how  memeory  has  been  partitioned  and  allocated  to  different 
section  of  the  computer  program(s)  and/or  data. 

Paragraph  3.4,  Object  Code  Creation:  This  paragraph  shall  describe  the  computer 
programs,  digital  processing  equipment,  hardware  and  procedures  required  to  be  uti¬ 
lized  to  generate  the  object  code. 
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a.  Computer  programs  required,  Including  but  not  limited  to  host  operating 
system,  compilers,  assemblers,  link  editors,  loaders,  and  translators. 

b.  Digital  processing  equipment  including  host  computer  hardware  minimum 
configuration,  vendor(s),  part  numbers,  and  other  pertinent  data  such  as  model  capacity 
and  special  capabilities. 

c.  Procedures  required  to  generate  object  coce  in  the  specified  media  suitable 
for  operation  in  the  target  microprocessor. 

Paragraph  3.5,  listings:  This  paragraph  shall  Include  source  and  object  code 
listings.  Comments  of  explanation  shall  be  included  and  considered  part  of  the 
listing.  The  level  of  detail  required  in  the  listings  shall,  when  used  with  other 
information  contained  within  tills  appendix,  permit  support,  modification  and/or 
adaptation  of  the  computer  programs  and  data  utilized  within  the  microprocessor. 

Section  4,  Quality  Assurance  Provisions:  Requirements  for  formal  verification 
of  the  microprocessor  and  associated  computer  program  requirements  of  Section  3 
and  Section  S  shall  be  included  in  this  section.  This  section  shall  reference  each 
performance, design  or  delivery  requirement  of  Section  3  and  Section  !>  and  specify  the 
specific  verification  method.  The  methods  of  verification  to  be  specified  herein 
may  include  inspection,  review  of  analytical  data  (analysis),  demonstrat ion (s ) ,  and 
test  (witnessing  test  results  and  review  of  qualitative  and  quantatitive  test  data 
resulting  from  formal  procedures,  controlled  environment  and  Instrumentation). 

This  section  shall  specify  requirement (s)  for  verification  to  the  level  of  detail 
necessary  to  establish  the  scope  and  accuracy  of  the  verification  method,  and  shall 
clearly  identify  each  verification  requirement  with  the  applicable  performance/design/ 
delivery  requirement  in  Sections  3  and  5. 

Paragraph  4.1  Microprocessor /Computer  Program  Testing:  This  paragraph  describes 
testing  at  the  microprocessor/computer  program-level  where  the  results  are  intended 
to  be  the  only  source  of  data  to  verify  specific  requirements  of  Section  3  or  5 . 

These  tests  may  be  conducted  prior  to  integration  of  the  microprocessor  into  the 
configuration  item. 


Paragraph  A, 2,  Configuration  Item  Testing:  This  paragraph  shall  identify  require¬ 
ments  specified  in  Section  3  which  will  be  verified  during  Configuration  Item -testing 
and  which  therefore  must  be  listed  as  Cl  test  requirements. 

Paragraph  4.3,  System  Test  Program;  This  paragraph  shall  identify  requirements 
specified  in  Section  3  which  cannot  be  verified  until  system  testing  (or  equivalent) 
and  which  therefore  roust  be  listed  as  system  test  requirements. 

Paragraph  A. 4,  Verification  Cross-Reference  Matrix:  This  paragraph  shall  include  a 
Verification  Cross-Reference  Matrix  identifying  each  Section  3  requirement  with  the 
Section  A  paragraph  where  the  qualification  requirement (s)  ls/are  specified. 

Section  6,  Preparation  for  Delivery:  When  applicable,  this  paragraph  shall  be 
used  to  describe  special  handling  requirements  such  as:  packaging  for  delivery  (e.g., 
to  ships  or  to  remota  sites),  which  may  require  special  labels,  etc.  This  para¬ 
graph  shall  also  specify  the  media  or  delivery,  e.g.,  cassettes,  cartridges,  magnetic 
tape,  punched  paper  tape,  punched  cards,  disks,  firmware,  etc.  Also  included  shall 
be  the  general  or  specific  characteristics  of  the  mod: a  as  required  for  qualification 
testing  and  verification.  If  special  or  unique  packaging  is  required  to  avert  possible 
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Preparation  Instructions  (Continuation) 

/ 

compromise  of  Cl  performance,  then  packaging  requirements  shall  be  included. 

t 

Section  6,  Notes:  This  section  of  the  specification  is  not  contractually  binding, 
and  should  not  contain  contractual  requirements.  It  may  Include  Information  or  particu¬ 
lar  importance  to  the  procuring  activity  in  using  the  specification  as  a  contractual 
instrument  or  administrative  or  background  information  (e.g.,  ordering  instructions 
for  technical  data  pertianing  to  the  computer  program,  or  specific  information  related 
to  the  use  of  the  program  in  future  assembly  and  integration  testing).  It  shall 
not  Include  requirements  which  constrain  design,  development,  or  qualification.  The 
text  may  be  preceded  with  the  statement  "Administrative  Information  Only  -  Not 
Contractually  Binding."  Background  information  or  rationale  which  will  be  of 
assistance  in  understanding  the  specification  itself,  may  be  included  herein.  The 
procuring  activity  must  specify  material  which  is  mandatory  for  inclusion  in  Section  6. 

3.  When  the  prime  item  is  for  training  equipment,  the  prime  item  product  fabrication 
specification  preparation  instructions  in  Appendix  VIII,  MIL-STD-490  shall  be 
Implemented  by  the  following  instructions: 

APPENDIX  VIII 

80.  TYPE,  Clb,  PRIME  ITEM  PRODUCT  FABRICATION  SPECIFICATION. 

80.1  Section  1,  "Scope."  No  Change 

Example: 

1.  Scope. 

1.1  "Scope."  This  section  shall  also  briefly  describe  the  intended  use 
of  the  trainer,  by  whom,  and  where  It  is  intended  to  be  used;  e.g.,  contractors' 
plant,  ATC  base,  etc. 

1.2  "Classification."  No  change. 

80.2  Section  2,  "Applicable  documents."  No  change. 

80.3  Section  3 ."Requirements 

80.3.1  Paragraph  3.1,  "Item  definition."  No  change 

80.3.1.1  Paragrpah  3.1.1,  "Major  component  list."  Add  "This  paragraph  shall 
list  the  major  items  of  equipment  required  to  operate  with  a  trainer  in  an  instruction 
situation  by  the  following  categories: 

"Contractor  furnished  system  operational  equipment." 

"Government  furnished  equipment  (by  expected  soruce)." 

80.3.1.2  Paragraph  3.1.2,  "Government  furnished  property  list."  No  change 

80.3.2  Paragraph  3.2,  "Characteristics."  No  change. 

80.3.2.1  Paragraph  3.2.1,  "Performance."  No  Change. 
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Preparation  Instructions  ^Continuation) 

80. 3. 3  Paragraph  3.3,  "Design  and  construction.' 


No  change. 


80.3.3.1  Paragraph  3.3.1,  "Production  drawings."  No  change. 

80.3.3.2  Paragraph  3.3.2,  "Standards  of  manufacturer."  No  change. 


80.3.3.3  Paragraph  3.3.3,  "Workmanship."  No  change 

80.3.4  Paragraph  3.4,  "F.uproduction  sample."  Not  required.  * 

80.4  Section  4,  "Quality  Assurance  Provisions."  No  change. 


80.4.1  Paragraph  4.1,  "General."  Add,  "For  trainers  requiring  extensive 
installation  and  checkout,  the  acceptance  Inspection  may  be  accomplished  at  the 
Installation  site.  This  does  not  necessarily  preclude  an  interim  acceptance  for 
specified  purposes  at  the  place  of  manufacturer." 


80.4.1.1  Parag^sph  4.1.1,  "Responsibility  for  inspection."  No  change. 

80.4.1.2  Paragraph  4.1.2,  "Special  tests  and  examination."  No  change. 

80.4.2  Paragraph  4.2,  "Quality  conformance  Inspections."  No  change. 

80.5  Section  5,  "Preparation  for  delivery."  Add, "Markings  for  items  appearing 

in  the  Training  Equipment  List  (TEL)  shall  coincide  with  their  identification  in  that 
document.  Interior  packages  and  exterior  shipping  container  shall  be  marked  in 
accordance  with  s peci fleet  ion  MIL-STD-129,  "Marking  of  Shipments,"  and  MIL-STD-130, 
Identification  Marking  of  U.  3.  Military  Property,"  and  the  words  "Training  Equipment" 
shall  be  stenciled  thereon.  Interior  packages  shall  also  bear  the: 


a.  TEL  item  number 

b.  Stock  number 


c.  Condition  status  (indicate  if  item  is  reject) 


Training  equipment,  including  part  of  trainers,  furnished  in  reject  status,  .shall  be 
permanently  marked  by  stencil,  stamp,  engraving,  or  decalcomania:  Factor' Production 
Reject. 

*  • 

80.6  Section  6,  "Notes."  No  change. 

80.6.1  Paragraph  6.1,  "Intended  use."  No  change. 

80.6.2  Paragraph  6.2,  "Ordering  data."  Not  required. 

80.7  Section  10,  "Appendix  I."  No  change. 
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1.  OBJECTIVE 


Agency  policy  frequently  dictates  the  use  of  standardization.  Given 
that  this  is  a  requirement,  evaluate  the  potential  for  utilizing  accredita¬ 
tion  of  computer  architectures  as  a  viable  tri -service  computer  acquisition 
strategy.  Determine  the  role  and  impact  of  MIL-STD-1750A,  MIL-STD-1862 
and  DoD  Instruction  5000. 5X  within  the  overall  defense  computer  acquisition 
standards . 

2.  SCOPE 

Each  service  has  been  acquiring  embedded  computer  systems  (versus 
commercial  ADP-type  computers)  utilizing  a  myriad  of  selection  criteria. 

The  end  user  is  often  required  to  implement  a  system  with  an  approved 
standard,  yet  technologically  obsolete,  computer.  Accreditation  has  been 
promoted  as  an  alternative  computer  acquisition  strategy. 

In  1978,  the  Navy's  Office  of  the  Assistant  Secretary  for  Research, 
Engineering  and  Systems  (ASN,  RE  § S)  assembled  a  panel  to  review  the  status 
and  trends  of  embedded  computer  applications  and  to  recommend  alternatives 
to  existing  acquisition  policy.  One  of  the  panel's  suggestions  was  the 
concept  of  accreditation,  which  had  the  following  goals: 

--  Stimulate  competition  in  production  (vs.  sole  source  production) 

--  Ease  technology  insertion 

--  Increase  flexibility  of  choice  for  program  managers 
--  Shorten  the  acquisition  cycle 
--  Minimize  cost  of  ownership 

With  this  as  background,  the  issues  relevant  to  the  objective  include: 
--  The  architectural  levels  at  which  accreditation  can  apply 
--  Hardware  and  software  logistics  support 


--  The  effect  of  multiple  vendors  on  the  attainment  of  standards 

--  Industry's  willingness  to  support  an  accreditation  policy 

--  The  capability  of  an  accreditation  policy  to  meet  service  operational 
and  technological  needs 

--  Other  influences  on  the  flexibility  of  choice  and  the  length  of  the 
acquisition  cycle. 

--  Administration  of  evaluation  and  selection  under  an  accreditation  policy 

--  The  introduction  of  new  architectures  as  standards 

As  interpreted  by  Panel  C,  all  computer  acquisitions  related  directly  to 
weapons  systems,  operational  support,  ATE,  and  the  like,  fell  within  its  scope. 
Hardware- intensive  processors  were,  however,  excluded.  The  only  HOL  considered 
at  the  source  level  of  architectural  standards  was  Ada.  Where  relevant,  a  given 
standard  was  assumed  to  apply  to  a  single  performance  envelope  (e.g.,  mini, 
maxi,  large  scale). 

3.  APPROACH 

Presentations  by  Lt.  Col.  Vance  Mall  of  the  Ada  Project  Office,  Doug  Stapp 
of  ASD/ENASD,  and  William  Smith  of  the  Office  of  the  Assistant  Secretary  of  the 
Navy  (R  ESS)  established  a  background  for  the  discussion  of  standardization 
and  accreditation.  See  Appendices  C  thru  !.. 

Early  in  the  panel's  discussions,  it  became  evident  that  the  concept  of 
accreditation  went  beyond  tire  definition  originally  used  by  the  Navy  (see 
second  sheet  of  Appendix  E) ,  which  has  viewed  it  in  contradistinction  to  its 
policy  of  standardization.  For  acquisitions  directed  toward  achieving  archi¬ 
tectural  standards  at  other  than  the  lowest  replaceable  unit  level,  accredi¬ 
tation  could  take  on  new  meanings.  Accordingly,  the  panel  decided  it  must 
provide  a  definition  to  frame  the  balance  of  its  deliberations.  This  definition 


(anticipating  the  next  section  of  this  report)  was 

"Accreditation  is  an  acquisition  strategy  by  which  a  product  is 

certified  to  be  suitable  for  service  use  in  accordance  with 

documented  criteria." 

With  this  definition,  it  became  clear  that  there  were,  in  fact,  several 
accreditation  approaches  that  needed  consideration.  Moreover,  the  panel 
decided  that  there  were  two  distinct  standardization  methods  that  have  been 
used  and  had  to  be  included  in  further  discussion.  These  acquisition  strategies 
are  defined  in  para  4.1. 

As  discussion  evolved,  it  became  evident  that  there  were  a  large  number 
of  issues  concerning  the  relative  merits  of  each  of  the  acquisition  strategies 
that  had  been  identified.  To  better  focus  on  each  of  these,  the  panel  decided 
to  evaluate  each  of  the  acnuisition  strategies  against  specific  criteria. 

These  criteria  (several  of  which  are  of  more  than  one  part)  were 

1.  Reliability  and  maintainability 

2.  Effect  on  logistics 

3.  Effect  on  personnel 

4.  Sources  for  development  and  qualification  funding 

5.  Promotion  of  a  competitive  environment 

6.  Effect  of  shortening  the  acquisition  cycle 

7.  Life  cycle  cost 

8.  Product  availability  --  marketplace 

9.  Capability  of  achieving  technological  currency 

Response  to  specific  weapons  systems  requirements  was  not  used  as  a 
criterion. 

The  panel  wished,  also, to  consider  the  effect  of  these  criteria  at 
four  levels  of  standards:  source,  object,  box,  and  LRU.  These  are  defined 
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in  Section  4  of  this  report.  Although  it  was  recognized  that  some  criteria 
would  apply  equally  to  two  or  more  of  these  levels,  it  was  clear  that  the 
evaluation  of  the  acquisition  strategies  vs.  criteria  vs.  standard  level  would 
be  a  formidable  task.  Accordingly,  to  make  the  evaluation  process  more 
manageable,  the  panel  divided  into  three  sub-panels,  with  each  responsible 
for  a  group  of  related  criteria.  Sub-Panel  1  tackled  criteria  1  thru  3, 
sub-Panel  2  dealt  with  criteria  4  thru  6,  and  sub-Panel  3  was  organized  to 
handle  criteria  7  thru  9. 

The  special  task  of  each  sub-panel  was  to  evaluate  tne  identified 
cquisition  strategies  (plus  a  sixth,  "laissez  faire,"  representing  the 
antithesis  of  standardization)  against  each  of  the  criteria  assigned  to  it 
and  at  each  applicable  standard  level  (Source,  et  at).  This  ’’ould,  in  effect, 
provide  an  in-depth  overview  of  the  entire  issue  of  standardization  and 
accreditation  independently  of  unique  service  needs. 

he  last  day  of  the  workshop,  each  of  the  sub-panels  reported  its  findings 
to  j  others.  This  enabled  all  panel  members  to  provide  input  to  the 
identification  of  the  key  issues,  and  to  draw  attention  to  the  need  for 
clarifi  ition  of  the  assumptions  underlying  each  sub-panel's  conclusions. 
Although  the  resultant  three-dimensional  array  of  mini -position  papers  is, 
in  itself,  a  set  of  conclusions,  before  adjourning,  the  panel  proceeded  to 
synthesize  the  work  accomplished  to  draw  some  general  conclusions  and 
recommendations . 
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4.  DISCUSSION 


The  bulk  of  this  section  is  formed  by  the  matrix  of  acquisition 
strategy/criteria/level  of  standard.  As  will  be  seen,  this  matrix  consists 
of  a  quick  reference  table  and,  of  greater  importance,  the  premises  and 
remarks  attending  all  but  the  most  obvious  entries  in  the  table.  Some 
definitions  and  additional  background  which  are  required  for  interpretation 
of  the  matrix  follow  in  paragraphs  4.1  thru  4.4. 

4 . 1  Acquisition  Strategies 

The  following  strategies  are  those  evaluated  by  the  panel.  Strategies 
4.1.2  thru  4.1.5  are  accreditation  techniques,  although  strategy  4.1.5  has 
also  been  regarded  as  a  standardization  method,  and  is  so  labeled. 

4.1.1  Laissez  faire  ...  So  named  by  the  panel  because  it  permits  each 
Program  Manager  (PM)  or  his  contractor  to  select  whatever  computer  architecture 
and  implementation  he  chooses.  This  is  the  de  facto  method  of  acquisition 
currently  used  for  many  programs.  Consideration  of  this  strategy  was  not  within 
the  scope  of  the  panel's  task.  it.  is  included  because  it  serves  as  a  reference 
(at  the  pole  opposite  that  of  4.1.6)  against  which  all  other  strategies  may  be 
compared. 

4.1.2  Accreditation  I  ...  In  this  strategy,  the  PM  has  the  responsibility 
for  verifying  (validating,  the  implementation  of  a  standard  architecture. 

This  technique  has  been  employed  by  the  Air  Force. 

4.1.3  Accreditation  II  ...  As  in  the  operation  of  a  qualified  parts  list, 
computer  manufacturers  develop  implementations  of  standard  architectures  and 
submit  them  to  the  government  for  validation  as  machines  qualified  to  join 
others  on  a  published  list. 

4.1.4  Accreditation  III  ...  In  this  strategy,  the  government  buys  a  large 
number  of  machines  for  application  to  one  or  more  programs;  in  essence,  acting 
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as  a  central  supplier  of  a  computer  commodity.  It  is  expected  that  the 
development  costs  of  each  such  implementation  of  a  standard  architecture 
would  involve  at  least  partial  government  funding.  This  strategy,  which 
has  been  considered  by  the  Navy,  lends  itself  to  multiple  developers  and 
producers,  with  the  number  of  developers  expected  to  be  greater  than  the 
number  of  producers. 

4.1.5  Standardization  I  ...  Still  an  accreditation  method,  in  this  strategy 
several  potential  suppliers  are  competitively  selected  and  funded  to  develop 
an  implementation  of  a  given  architecture.  One  of  these  is  subsequently 
awarded  a  production  contract.  A  second  source  production  contract  is  also 
possible.  This  strategy  is  now  employed  by  the  Army  and  Navy. 

4.1.6  Standardization  II  ...  A  single  developer  is  funded  to  develop  and 
produce  machines  conforming  to  a  given  architecture.  That  is,  a  single  supplier 
is  awarded  a  contract,  normally  on  a  competitive  basis,  for  both  development 
and  production.  Second  sourcing  is  possible  here,  also. 

4.2  Architectural  Levels  to  Which  Standards  Apply 

Four  levels  were  defined.  It  should  be  noted  that  for  certain  criteria, 
the  level  was  immaterial,  as  will  be  seen  in  the  matrix,  while  for  others, 
it  was  of  critical  importance. 

4.2.1  Source  ...  This  describes  the  ability  to  deliver  a  computer  and  (if 
not  capable  of  direct  execution  of  HOL  programs)  a  compiler  such  that  HOL 
programs  will  run  predictably.  The  transportability  of  assembly  language 
programs  is  not  an  issue  here. 

4.2.2  Object  ...  Implementation  of  an  object  level  standard  refers  to 
transportability  of  the  assembled  or  compiled  source  language.  This  may  be 
viewed  as  a  run-time  architectural  standard.  HOL  standards  are  implicit. 
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4.2.3  Box  ...  This  is  a  hardware  standard  at  the  level  of  form,  fit,  and 
function.  1I0L  and  ISA  standards  are  implicit. 

4.2.4  LRU  ...  Similar  to  box  level  except  that  the  implementation  specifics 
are  standard  as  well  for  logistics  commonality.  That  is,  computers  at  this 
level  of  standards  conformance  are  spared  identically.  HOL,  ISA,  and  box 
standards  are  implicit. 

4 . 3  Weighting  of  Criteria 

In  general,  the  several  criteria  were  not  rated  with  regard  to  relative 
importance,  although,  as  will  be  seen,  certain  criteria  were  weighted  with 
respect  to  other  criteria.  The  reason  weighting  was  not  attempted  derives 
from  the  recognition  that  no  weighting  scheme  can  apply  equally  to  all  system 
applications.  For  example,  hardware  logistics  would  be  much  less  important  for 
space  probe  applications  than  it  would  for  applications  supported  by  recurring 
depot  level  maintenance.  Differences  among  the  four  services  in  their  support 
philosophies  would  also  affect  weighting.  A  weighting  scheme  of  suitable 
generality  would  require  considerably  more  study  than  could  be  undertaken 
within  the  time  constraints  of  the  workshop. 

4 . 4  Interpretation  of  Tabular  Information 

The  tables  found  later  in  this  section  summarize  the  issues  attending 
the  evaluation  of  the  criteria  against  the  six  strategies.  For  most  of  the 
criteria,  the  evaluation  is  made  at  the  four  levels  of  standardization  with 
the  key  to  the  level  to  which  each  evaluation  applies  given  in  the  upper  left 
hand  corner  of  each  table.  The  numbers  accompanying  the  ratings  (very  good 
to  very  poor)  refer  to  the  notes  following  each  table. 

The  evaluat ion  rat ings  given  (very  good  to  very  poor)  must  be  used 
guardedly .  Indeed,  several  of  the  panel  stated  that  they  would  prefer  the 
report  not  include  these,  for  fear  that  if  quoted,  reproduced,  or  summarized 
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out  ot'  the  context  of  the  accompanying  premises  and  remarks  (Notes  1  thru  67) , 

the  ratings  could  lead  to  inferences  of  doubtful  correctness  or  value.  The  \ 


tabular  ratings  are  presented  only  as  a_  means  of  providing  a_  synoptic  picture 


of  the  influences  exerted  by  each  of  the  strategies . 
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4 . 5  Discussion  Specific  to  the  Criteria  of 


Reliability  and  Maintainability 
Software  Logistics 
Hardware  Logistics 
Military  Personnel 
Civilian  Personnel 

4.S.1  Amplification  of  Evaluation  Criteria  -- 

Reliability  and  Maintainability:  These  were  evaluated  relative  to 
their  effect  on  the  availability  of  a  computer  to  meet  mission  requirements 
and  the  ease  of  maintaining  computers  in  the  presence  of  failures.  This 
criterion,  then,  measures  the  government's  expectations  of  reliability  under 
each  of  the  acquisition  strategies  and  standardization  levels. 

Software  Logistics:  This  criterion  was  evaluated  with  respect  to  the 
effect  on  support  software,  to  the  exclusion  of  the  transportability  of 
applications  software. 

Hardware  Logistics:  This  is  within  the  context  of  the  existing  sparing 
and  maintenance  philosophies  of  the  services. 

Military  Personnel:  This  relates  to  the  impact  on  die  technicians  and 
system  operators  in  the  field.  Tiiat  i~,  it  reflects  the  potential  availability 
of  the  talent  pool  necessary  to  support  the  system  us  well  as  the  level  of 
training  required  to  ensure  tiie  capability  of  maintaining  and  operating  the 
system. 

Civilian  Personnel:  This  factor  measures  the  availability  of  the 
necessary  pool  of  talent  within  industry  --  engineers,  designers,  and  techni¬ 
cians  --  able  to  respond  to  a  particular  acquisition  strategy.  Civilian 
contractors  hired  to  provide  services  to  the  government  were  not  considered 


to  be  an  issue. 


4.5.2  General 


1.  Sub-Panel  1,  to  whom  the  above  criteria  were  assigned,  felt  that 
since  the  laissez  faire  approach  to  acquisition  does  not  have  associated 
levels  of  standardization  appropriate  to  it,  and  is,  in  any  case,  an  unaccept¬ 
able  method,  it  would  evaluate  laissez  faire  without  regard  to  levels  of 
standardization,  but  use  it  merely  as  a  basis  of  comparison. 

2.  The  Accreditation  I  approach  was  evaluated  for  standardization  only 
at  the  source  level  and  at  the  object  level.  It  really  is  valid  only  at  the 
object  level  since  the  basis  of  the  approach  is  an  ISA  standard  and  there  is 
no  attempt  to  standardize  below  the  ISA. 

4.5.5  Premises  -- 

1.  It  was  assumed  by  the  sub-panel  that  standardization  at  the  LRU  level 
included  standardization  at  the  box  and  object  levels.  Likewise,  standardiza¬ 
tion  at  the  box  level  was  assumed  to  imply  standardization  at  the  object  level. 

2.  It  was  originally  assumed  by  the  sub-panel  that  source  level  standard¬ 
ization  was  only  viable  when  high  order  language  (HOI)  direct  executable  machines 
were  within  the  state  of  the  art.  After  discussions  with  sub-Panel  3,  it  was 
accepted  by  sub-Panel  1  that  source  level  standardization  we  lid  mean  that  the 
contractor  would  deliver  a  computer  and  compiler  and  that  HOL  programs  would 

run  predictably  on  the  machine  and  ISA  or  object  level  programs  would  not  be 
transportable. 

3.  It  was  further  assumed  by  the  sub-panel  that  standardization  at  the 
source  level  means  that  multiple  code  generators  would  be  required  for  target 
machines  since  ISA's  would  not  be  standard.  Standardization  at  the  object 
level  implied  CPU  ISA  standardization  and  no  I/O  standardization  of  any  kind. 
Standardization  at  the  LRU  or  box  level  implies  I/O  standardization  and  the 


equivalent  transportability  of  the  HOL  and  object  level  software. 

4.  In  terms  of  sparing  philosophy,  it  was  assumed  by  the  sub-panel 
that  when  standardization  was  levied  at  the  box  level,  the  services  would 
also  replace  and  spare  at  the  box  level,  thereby  requiring  personnel  of 
lesser  skill  to  maintain  the  equipment.  This  is  in  contrast  to  standardiza¬ 
tion  at  the  LRU  level  where  it  was  assumed  that  the  services  would  also  replace 
and  spare  at  the  LRU  level. 

5.  It  was  assumed  that  major  improvements  in  reliability  would  result 
from  the  ability  of  contractors  to  use  more  current  technology  in  the  computer 
design.  It  was  also  recognized  that  major  improvements  would  also  occur  during 
a  long  production  run  with  a  fixed  technology  as  a  result  of  learning  curve 
experience  and  reliability  improvement  programs;  however,  no  attempt  was  made 
to  further  develop  the  advantages  of  one  approach  over  the  other. 

4.5.4  Ratings  Table  --  See  Figure  4-1. 

4.5.5  Notes  applicable  to  Rating  Table  -- 

1.  RAM/Source  and  object/Accreditation  I  5  II:  Accreditation  I  §  II  do 
not  inhibit  technology  infusion  by  the  contractors;  therefore  it  was  determined 
that  reliability  and  maintainability  benefits  would  be  increased. 

2.  Military  personnel/object/Accreditation  I  5  II:  Under  these  strategies, 
and  implementing  ISA  standardization,  hardware  commonality  would  be  non-existent 
and  it  would  be  more  difficult  for  the  military  to  provide  the  personnel 
necessary  to  maintain  and  operate  the  systems. 

3.  Civilian  personnel/object,  box,  LRU/Accreditation  I  5  II:  No  compari¬ 
son  with  laissez-faire  was  intended.  The  generally  low  rating  is  due  primarily 
to  the  unavailability  of  specialized  engineering/design  personnel  within  the 
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civilian  community  to  be  committed  to  the  developments  of  specialized 
computers  to  meet  military  specified  ISA's/HOL's. 

4.  Source/Accreditation  II:  Under  the  strategy  of  Accreditation  II, 
technology  used  to  build  a  computer  is  completely  flexible  with  standardi¬ 
zation  at  the  source  level.  Reliability  would  be  high,  software  logistics 
would  be  average  because  of  the  number  of  code  generators  that  would  be 
required  for  various  target  machines,  hardware  logistics  would  be  extremely 
low  due  to  the  different  technologies  used  by  the  multiple  vendors,  military 
personnel  would  be  extremely  low  because  of  the  training  required  to  maintain 
multiple  computers.  Civilian  personnel  impact  would  be  very  favorable  because 
of  the  improved  productivity,  especially  if  a  high  order  language  was  selected 
by  the  military  that  was  also  in  common  use  in  commercial  applications. 

5.  RAM  S  Software  Logistics/box/Accreditation  II:  The  high  rating  is  a 
result  of  the  fact  that  technology  flexibility  is  completely  available  to  the 
contractor.  In  addition,  the  box  level  of  standardization  favorably  impacts 
software  logistics  since  it  implies  both  ISA  and  I/O  standardization  as  well. 

6.  Hardware,  logistics/box/ Accreditation  II:  In  this  case,  hardware 
logistics  is  affected  by  the  additional  cost  and  space  required  to  support  a 
maintenance  and  sparing  philosophy  required  when  there  are  multiple  vendors 
for  the  same  computer  using  different  technologies.  This  approach  also 
usually  requires  depot  level  repair  as  opposed  to  organization  or  intermediate 
level  repair. 

7.  Hardware  logistics/LRU/Accreditation  II:  The  high  benefi  :  is  due  to 
the  fact  that  multiple  manufacturers  are  producing  standard  LRU's.  Therefore, 
the  only  risk  is  that  of  interchangeability  of  spares. 


8. 


Hardware  logistics/LRU/Accreditation  II:  The  strategy  of  open 


accreditation  would  have  most  favorable  benefit  with  box  level  standardiza¬ 
tion  for  military  personnel  because  sparing  would  be  at  the  box  level  and 
replacement  would  be  likewise  at  the  box  level.  Therefore,  trouble  shooting 
would  involve  only  detecting  a  failed  computer  and  replacing  that  computer 
with  a  spare. 

9.  Military  personnel/LRU/Accreditation  II:  The  impact  in  this  area 
is  primarily  dependent  upon  the  adequacy  of  the  BIT.  If  BIT  can  be  expected 
to  exceed  99°6  isolation  to  one  LRU,  the  benefit  of  this  approach  is  definitely 
improved. 

10.  RAM/Source,  object,  box/Accred  itation  III  f]  Standardization  I: 

These  generally  high  ratings  are  given  because  it  is  assumed  that  when  more 
than  one  developer  was  involved,  the  competition  will  provide  the  government 
a  better  technical  product  using  the  latest  technology  and  thereby  improved 
reliability  at  the  time  of  development. 

11.  RAM/LRU/Accreditation  III,  Standardization  I,  Standardization  II: 

The  lower  ratings  were  assessed  because  constraining  the  developer  to  standardize 
at  an  LRU  level  would  limit  his  flexibility  in  terms  of  utilizing  the  latest 
technology  to  achieve  major  reliability  improvements. 

12.  Military  personnel/Accreditation  III:  Under  the  assumption  that 
standardization  at  the  box  levels  implies  sparing  at  the  box  level,  the  impact 
on  military  personnel  would  be  the  most  favorable.  LRU  standardization  would 
be  somewhat  less  of  a  benefit  because  there  would  be  a  requirement  for  more 
skilled  personnel  coupled  with  the  risk  of  non-interchangeability  of  LRU's 
between  multiple  vendors.  The  average  benefit  with  standardization  at  the 
source  or  object  level  is  due  to  the  fact  that  multiple  vendors  would  imply 
multiple  logistics  problems. 
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13.  Civilian  personnel/Accreditation  III  $  Standardization  I: 

Relative  to  a  strategy  involving  N  developers,  the  impact  of  civilian 
personnel  is  primarily  in  the  development  stage  since  it  could  in  fact 
require  N  times  as  many  engineers/designers,  technicians,  resulting  in  a 
general  unwillingness  of  industry  to  participate. 

14.  RAM/Source,  object,  box/Standardization  II:  The  strategy  of  one 
developer  -  one  producer  does  not  promote  improved  reliability  through 
insertion  of  major  technology  advances  through  frequent  competitions. 

However,  this  would  be  offset  by  a  reliability  improvement  program  that 
would  allow  improvements  to  be  implemented  during  a  long  production  cycle. 

15.  Software  logistics/object,  box,  LRU/Standardization  II;  Standardi¬ 
zation  at  the  object,  box  or  LRU  level  would  in  effect  give  the  government 

a  defacto  software  standard. 

16.  Hardware  logistics/Standardization  II:  With  a  strategy  of  one 

developer  -  one  producer  it  does  not  really  make  any  difference  what  the 

level  of  standardization  is  because  the  hardware  itself  becomes  a  defacto 

standard  at  the  LRU  level.  However,  within  a  concept  of  standardization  above 
the  LRU  level,  the  contractor  could  have  the  option  of  hardware  modifications 
over  the  length  of  the  production  run  which  may  cause  differences  at  the  LRU 
level.  The  significance  of  this  impact  would  be  dependent  upon  the  configura¬ 
tion  management  controls  that  were  imposed  on  the  contractor  by  the  government. 

17.  Military  pcrsonncl/source  H  object/Standardization  II:  Standardizing 
at  the  source  or  object  level  is  good  for  the  military  because  the  training 
and  maintenance  is  the  same  for  all  personnel  across  all  systems  applications. 

18.  Civilian  personnel/source/Standardization  II:  Although  the  impact 
is  favorable,  it  was  felt  that  industry  would  not  participate  unless  it  had 
the  specific  expertise  required  to  develop  a  computer  at  a  military  specified 
1101.  level  of  standardization. 


19.  Civilian  personnel/object/Standardization  II:  Note  18  again 
applies  except  that  it  will  he  even  more  difficult  for  industry  to  find  the 
expertise  necessary  to  participate  when  it  is  also  standardizing  on  a 
military  specified  ISA. 

20.  Military  5  Civilian  personnel/box  f,  I.RO/Standardization  II:  It 
was  considered  that  standardization  at  the  lowest  level,  and  using  the 
acquisition  strategy  of  one  developer  and  one  producer,  was  easiest  for 
everyone  concerned,  military  and  civilian. 

21.  Civilian  personnel/source/Accreditation  I:  A  high  rating  is  based 
on  the  assumption  that  a  HOL  adopted  by  the  Dob  would  be  compatible  with  a 
HOL  used  for  commercial  applications  and  supported  by  computer  manufacturers 
in  that  regard. 

4.5.6  Conclusions  -- 

1.  Except  for  reliability  and  maintainability,  the  more  restrictive 
acquisition  strategies  (moving  from  laissez  faire  to  standardization  II) 
favorably  improve  the  impact  that  the  acquisition  strategy  has  on  the  measures 
of  comparison.  Further,  increasing  the  level  of  standardization  (moving  from 
source  to  LRU)  also  has  a  favorable  effect  on  the  criteria  considered.  That 
is,  the  more  restrictive  the  acquisition  strategy  and  the  greater  the  level  of 
standardization,  the  greater  would  be  the  benefits  to  the  government. 

2.  RAM  is  the  exception  to  the  first  conclusion.  The  reason  is  that  the 
less  restrictive  the  acquisition  strategy,  the  greater  the  potential  for 
contractors  to  be  innovative  in  using  the  most  current  technology  to  achieve 
improved  levels  of  reliability.  The  attendant  ability  to  implement  extensive 
BIT  using  newer  technologies  also  improves  maintainability. 


4.6  Discussion  Specific  to  the  Criteria  of 


Funding  Sources 
Competition 
Acquisition  Cycle 

4.6.1  Amplification  of  Evaluation  Criteria  -- 

Funding  Sources:  This  refers  to  the  willingness  of  industry  to 
invest  in  computer  development  and  qualification. 

Competition:  Competition  was  evaluated  in  terms  of  the  ability  of 
an  acquisition  strategy  to  promote  a  competitive  environment. 

Acquisition  Cycle:  This  criterion  is  the  expected  effect  of  an 
acquisition  strategy  on  the  lengthening  or  shortening  of  the  acquisition 
cycle  of  a  weapons  system  (not  of  the  computer) . 

4.6.2  General  -- 

Although  production  quantity  had  initially  been  considered  as  a  separate 
criterion,  sub-Panel  2  discovered  that  quantity  considerations  were  implicit 
in  the  discussions  of  the  other  criteria. 

4.6.3  Premises  -- 

1.  DoD  market  stability  (commitment)  is  critical  to  industry  investment 
and  to  attracting  competition  (i.c.,  avoiding  market  fragmentation). 

2.  It  is  essential  that  both  development  and  production  be  linked 
together  to  provide  industrial  incentive.  A  production  allocation  plan  is 
critical  to  achieving  multiple  production  suppliers,  as  in  Accreditation  III. 

3.  DoD  microprocessor  needs  may  promote  different  industry  incentives 
and  lead  to  other  acquisition  strategies. 

4.  Actual  and  perceived  objectivity  and  fairness  are  critical  to  any 
DoD  acquisition  policy  for  industry  acceptance  (i.e.,  willingness  to  complete 
and  invest) . 


5.  In  general,  the  attraction  of  industry  investment  was  assumed 
to  be  good  for  the  government.  Thus,  high  ratings  in  this  column  of  the 
rating  table  (Figure  4-2)  translate  into  a  tendency  for  industry  to  invest 
under  the  variolas  standardization/investment  strategies.  Conversely,  a  low 
score  should  be  interpreted  as  a  tendency  for  industry  not  to  invest,  and  one 
should  expect  government  funding  to  dominate. 

6.  Sub-Panel  2  found  the  level  of  standardization  (source  et  al)  to 
be  immaterial  to  the  criteria  of  para  4.6.  Accordingly,  Figure  4-2  does 
not  depict  a  "third  dimension"  to  account  for  the  four  levels. 

4.6.4  Ratings  Table  --  See  Figure  4-2 

4.6.5  Notes  Applicable  to  Ratings  Table  -- 

22.  Controversial  findings:  In  the  discussions  with  the  entire 
Panel  C  membership,  substantial  disagreement  emerged  regarding  the  degree 
of  industrial  concentration  which  would  actually  take  place.  One  school  of 
thought  held  that  industry  will  respond  to  the  certain  knowledge  that  its 
participation  v:ll  be  rewarded  (as  in  Standardization  II),  while  the  other 
school  believed  that  the  greater  the  opportunity  to  make  a  sale  (laissez 
faire  as  the  extreme),  the  greater  will  be  industry's  willingness  to  risk 
investment . 

23.  Funding  sources:  The  laissez  faire  and  Accreditation  II  strate¬ 

gies  tend  to  discourage  investment  because  the  market  is  too  diffuse,  lacks 
certainty  .  .■  j.rec1-  ,  ,  and  does  not  lend  itself  to  accurate  return  on 

investment  assessments.  The  two  standardization  strategies  tend  to  draw 
the  heaviest  investment  because  the  market  is  concentrated  and  defined, 
because  of  the  implied  .  commitment,  because  risks  are  bounded,  and 
because  the  business  opportunity  (return  on  investment)  is  more  quantifiable. 
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Accreditation  I  and  III  were  rated  in  between  these  boundaries,  with 
Accreditation  III  having  a  slight  edge  over  Accreditation  I  in  market 
certainty  and  service  commitment. 

24.  Competition:  The  laissez  faire  strategy  was  seen  as  being  the 
least  advantageous  in  fostering  a  broad  competitive  environment,  primarily 
because  it  encourages  concentration  to  a  few  (or  perhaps  one)  firm  willing 
or  able  to  bear  marketing  and  development  burden  across  many  different 
government  program  managers  and  system  prime  contractors.  The  barriers  to 
entry  are  high  and  the  "staying"  costs  are  also  high.  This  argument,  too, 
drew  some  disagreement  in  the  full  Panel  C  deliberations  and  no  resolution 
of  differences  was  achieved. 

Accreditation  II  was  thought  to  experience  the  same  competition  pheonmenon, 
but  not  quite  as  severely  as  the  laissez  faire  strategy.  The  two  standardi¬ 
zation  strategies  do  not  suffer  from  this  in  the  initial  competition,  and 
hence  were  judged  best  on  this  factor.  However,  the  issue  of  subsequent 
competition  was  not  fully  addressed. 

25.  Acquisition  cycle  time:  This  factor  did  not  appear  to  be  a 
significant  driver  in  determining  a  computer  acquisition  strategy.  Never¬ 
theless,  for  completeness,  the  sub-panel  did  a  relative  ranking.  The 
laissez  faire  strategy  holds  the  most  potential  danger  because  each  develop¬ 
ment  is  a  "new  game,"  and,  as  such,  has  the  potential  for  impact  on  acquisi¬ 
tion  cycle.  At  the  other  pole,  the  two  standardization  strategies  had  the 
most  favorable  impact  on  this  factor  because  products  would  be  more  or  less 
instantly  available  once  development  was  completed.  The  three  accreditation 
strategies  were  viewed  as  only  slightly  less  advantageous  because  some  choice 
is  involved  and  it  was  reasoned  that  this  choice  process  would  consume  some 
time. 
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4.6.6  Conclusions  - 


1.  Industrial  investment  is  more  likely  to  be  drawn  toward  standardi¬ 
zation  strategies  than  to  the  more  flexible  strategies*. 

2.  Standardization  strategies  tend  to  promote  a  slightly  more  competi¬ 
tive  environment  than  do  accreditation  strategies;  laissez  faire,  least  of 
all. 

3.  Standardization  and  accreditation  strategies  have  about  the  same 
impact  on  system  acquisition  time,  and  both  appear  better  than  laissez  faire. 
This  factor  does  not  seem  to  be  a  driver  in  determining  a  computer  acquisition 
strategy. 

4.  The  government  members  of  sub-Panel  2  were  asked  to  respond  to  the 
question:  "What  is  the  most  important  of  these  criteria  with  respect  to  their 
use  in  evaluating  the  various  strategies?"  Their  unanimous  response  was  the 
order  of  importance  of  Competition,  Funding  Sources,  and  Acquisition  Cycle. 


‘Disagreement  exists  depending  on  the  degree  of  industry  concentration/ 
investment  which  will  occur  or  is  anticipated  to  occur. 
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4 . 7  Discussion  Specific  to  the  Criteria  of 


Life  Cycle  Cost 
Product  Availability 

Capability  of  Achieving  Technological  Currency 
4.7.1  Amplification  of  Evaluation  Criteria  -- 

Life  Cycle  Cost  was  divided  into  5  categories: 

•  Hardware  Development  Cost  --  The  steady  state  aggregate  cost  to 
the  DoD  of  the  development  of  all  embedded  computer  systems. 

•  Hardware  Production  Cost  --  Cost  to  the  DoD  of  embedded  computers 
acquired  in  production  quantities, 

•  Hardware  Logistics  Cost  --  Cost  of  maintaining  embedded  computers. 

•  Software  Development  Cost  --  Includes  cost  of  tool;,  training,  and 
code  production. 

•  Software  Logistics  Cost  --  Software  maintenance. 

Product  Availability  was  divided  into  2  categories: 

•  Assurance  of  Adequate  Quantities  --  The  level  of  conridenee  that  the 
DoD  can  procure  the  necessary  quantities  independent  of  the  commercial 
climate,  for  an  equitable  price,  for  the  expected  production  life  of 
the  product. 

•  Adequate  Industry  Participation  --  Industry's  willingness  to  invest, 

as  a  function  of  competition.  Compare  with  para  4.6.1  (Funding  Sources). 
Technological  Currency:  This  is  a  measure  of  the  degree  to  which  the 
latest  available  technology  is  captured  in  the  computer  acquired  for  any  given 
weapon  system. 
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4.7.2  General 


Faced  with  the  forbidding  task  of  evaluating  8  unique  criteria,  sub- 
Panel  3  decided  to  work  first  at  the  object  level  of  standards  and  then 
perturb  those  results  as  appropriate  to  the  source,  box  and  LRU  levels.  As 
it  developed,  it  was  necessary  to  treat  the  box  level  hastily,  and  the  LRU 
not  at  all  as  time  ran  out. 

4.7.3  Premises  -- 

1.  Object  level  standardization  was  taken  to  mean  that  computer  archi¬ 
tectures  would  be  restricted  to  those  on  a  list  such  as  proposed  under  5000. 5X. 
Currently  accepted  architectures  were  used  in  considering  scenarios.  At  this 
and  other  levels  of  standards,  transient  effects  caused  by  the  institution  of 
standards  were  disregarded.  Steady  state  comparisons  were  sought,  in  spite 

of  some  strong  feelings  that  nothing  stands  still  long  enough  to  achieve  a 
steady  state. 

2.  The  average  production  cost  for  military  computers  is  driven  by 
many  factors,  but  it  was  agreed  by  the  sub-panel  that  the  major  discriminant 
between  acquisition  strategies  is  the  length  of  productions  runs;  or,  conversely, 
the  degree  to  which  the  market  is  split  up  between  different  producers.  The 
strategies  also  differ  as  to  how  effectively  the  production  price  can  be 
competed  and  thereby  driven  down. 

3.  Hardware  logistics  cost  is  driven  chiefly  by  the  number  of  different 
computer  types  in  service,  each  with  its  own  parts  list.  Some  cost  penalty 
will  be  experienced  when  one  type  is  produced  by  multiple  sources,  leading  to 
problems  with  inter-changeability  of  parts.  The  sub-panel  noted  that  the 
entire  logistics  problem  is  mitigated  by  higher  levels  of  circuit  integration. 
VLSI  brings  such  a  reduction  in  number  of  replaceable  parts  (with  concurrent 
increase  in  reliability)  that  the  contribuiton  of  hardware  logistics  to  life 
cycle  cost  will  diminish  sharply  in  the  years  ahead. 


4.  In  an  unconstrained  environment,  each  different  architecture 
requires  its  own  set  of  software  tools,  including  assembler,  compilers, 
simulator,  and  all  elements  of  the  environment.  Because  of  limitations 
on  development  dollars,  a  complete  software  development  environment  can 
only  be  afforded  for  a  few  architectures.  For  other  architectures, 
inadequate  tools  will  mean  high  cost  of  application  software.  Architectural 
proliferation  also  increases  training  costs,  and  particularly  increases  the 
cost  of  all  code  which  must  be  written  in  native  language  and  assembled, 
which,  at  minimum,  includes  all  I/O  oriented  code.  When  the  number  of 
architectures  is  constrained,  the  foregoing  cost  problems  are  controlled. 
Software  costs  are  architecture  dependent  and  are  not  affected  by  the  manner 
or  variety  of  implementation. 

5.  Assurance  that  production  quantities  will  be  available  immunizes 
tne  government  against  industrial  instability.  It  is  driven  entirely  by  the 
number  of  sources  available.  Sub-Panel  3  did  not  see  this  as  a  strong 
discriminant  among  acquisition  strategies. 

6.  Adequate  industry  participation  is  a  measure  of  the  disadvantages 
of  highly  competitive  strategies  as  viewed  by  industry.  As  the  investment 
required  to  compete  goes  up  and/or  the  chance  of  winning  some  profitable 
business  goes  down,  it  becomes  less  attractive  for  industry  to  get  in  the 
game.  The  sub-panel  noted  that  in  the  case  of  each  of  the  AN/UYK-43  and 
AN/UYK-44  competitions,  only  two  companies  perservered  to  the  point  of 
submitting  proposals.  The  others  dropped  out  for  just  the  reasons  cited. 

As  the  congress  urges  more  strongly  competitive  strategies,  this  is  a 
significant  criterion  to  bear  in  mind. 

7.  Technological  currency  is  a  measure  of  the  degree  to  which  the 
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latest  available  technology  is  captured  in  the  computer  acquired  for 
any  given  weapon  system.  Generally,  it  is  high  for  computers  developed 
specifically  for  a  weapon  system,  less  when  the  Program  Manager  has  to 
procure  through  a  central  buy,  which  may  not.  be  a  recent  development. 

Sub-Panel  3  did  not  regard  this  criterion  as  a  major  one. 

8.  With  regard  to  hardware  development  costs  at  the  source  level 
of  standards,  the  sub-panel  agreed  that  development  costs  would  be  increased 
to  the  extent  new  architectures  were  brought  into  play,  but  reduced  to  the 
extent  that  militarization  of  commercial  machines  would  be  permitted.  The 
latter  option  becomes  increasingly  attractive  as  the  level  of  circuit  inte¬ 
gration  increases. 

4.7.4  Ratings  Table  -- 

See  Figures  4-3  and  4-4.  Recall  that  notes  applying  to  the  object  level 
of  standards  apply  equally  to  the  other  levels  unless  noted  otherwise.  Where 
two  ratings  are  separated  by  a  solidus  (/),  the  first  applies  to  single  source 
and  the  second  to  second  source  utilization. 

4.7.5  Notes  Applicable  to  Rating  Table  -- 

26.  Hardware  development  cost/object  level:  This  was  seen  as  being  gen¬ 
erally  proportional  to  the  number  of  different  computers  being  developed  over 
a  given  period  of  time.  The  sub-panel  also  agreed  that,  since  new  architec¬ 
tures  usually  require  some  clean-up  during  the  first  implementation,  standards 
which  discourage  proliferation  of  new  architectures  tend  to  lower  development 
costs.  Acquisition  strategies  that  are  predicated  on  contractor  financing  for 
development  result  in  minimum  cost  to  the  DoD . 


-23- 
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27.  Hardware  Development/object/laissez  faire:  This  unconstrained 
approach  leads  to  the  largest  number  of  different  computer  developments. 

28.  Hardware  Development/object/Accreditation  I:  Here  again,  the 
tendency  is  to  develop  a  new  computer  for  each  new  platform  or  weapon  system. 

29.  Hardware  Development/object/Accreditation  II:  Development  costs  are 
incurred  by  the  contractor,  so  DoD  sees  minimal  cost. 

30.  Hardware  Development/object/Accrcditation  111  and  Standardization  1: 
Central  buy  strategies  preclude  new  developments  until  they  are  clearly  life 
cycle  cost  justified,  keeping  development  costs  low.  Multiple  developers  cost 
more  than  single  developers. 

31.  Hardware  Development/object/Standardization  II:  Similar  to  note  30, 
but  one  developer  costs  less  than  multiple  developers. 

32.  Hardware  Development/object/Standardization  I  8  II  witli  second  sourc¬ 
ing:  An  additional  increment  of  development  costs  is  required  to  develop  and 
qualify  a  second  source. 

33.  Hardware  Production/object/laisscz  faire,  Accreditation  I,  and 
Accreditation  II:  These  strategies  permit  a  larger  number  of  types  and 
therefore  shorter  runs  than  the  central  buy  strategies. 

34.  Hardware  Production/object/Accreditation  III:  Although  this  was 
ranked  very  good,  the  ranking  relative  to  Standardization  I  is  a  function  of 
market  size.  Central  buy  minimizes  the  number  of  different  types,  and  multi¬ 
ple  developers  compete  the  production  price.  Breaking  up  the  run  among  an 
even  greater  number  of  producers  decreases  the  advantages  of  volume  production 
but  permits  continuing  competition  for  follow-on  market  share.  The  latter  only 
predominates  for  very  large  total  buys. 

35.  Hardware  Production/object  Standardization  1:  This  has  all  the 
advantages  of  long  runs  and  competition  for  production. 


36.  Hardware  production/object/Standardization  II:  This,  too,  has  all 
the  advantages  of  long  runs,  but  no  competition  after  development. 

37.  Hardware  production/object/Standardization  I  5  II  with  second 
sourcing:  A  small  increment  of  cost  is  added  for  this.  This  results  from 
a  reduction  of  production  volume  and  the  added  costs  of  coordination  and 
parts  compatibility. 

38.  Hardware  logistics/object/laissez  faire:  The  maximum  number  of 
computer  types  generates  the  maximum  logistics  cost. 

39.  Hardware  logistics/obj ect/Accreditation  I  5  II:  Less  proliferation 
of  types  than  laissez  faire,  but.  more  than  central  buy  strategies. 

40.  Hardware  logistics/object/Accreditation  III:  This  has  minimal 
proliferation  of  types,  but  includes  some  avoidable  costs  traceable  to  the 
difficulties  of  multiple  sources. 

41.  Hardware  logistics/object/Standardization  I  5  II:  These  have 
minimum  proliferation  of  types.  With  second  sourcing,  there  is  a  small 
additional  cost  due  to  difficulties  of  multiple  sources. 

42.  Software  dev»iopment/object/laissez  faire:  Architectural  prolif¬ 
eration  leads  to  high  cost. 

43.  Software  development/object/all  but  laissez  faire:  Costs  are 
improved  when  architectural  proliferation  can  be  controlled. 

44.  Software  logistics/object:  Software  maintenance  can  be  viewed  as 
"in-service  development,"  so  logistics  costs  run  exactly  parallel  to  develop¬ 
ment  costs.  Proliferation  has  an  acute  effect,  because  specialists  cannot 

be  afforded  to  handle  the  whole  inventory. 

45.  Quantities/object/laissez  faire:  Only  one  source  per  computer. 

46.  Quantities/object/Accreditation  I:  Within  the  context  of  a  single 
source  per  computer,  this  was  rated  average.  However,  the  sub-panel  noted 
that  second  sourcing  is  applicable  to  this  category  and  would  raise  the 


-25- 


availability. 

47.  Quantities/object/Accreditation  II:  The  feeling  was  that  this 
strategy,  if  successful,  would  generate  sufficient  redundancy  of  suppliers 
to  cushion  any  failures  among  them. 

48.  Quantities/object/Accreditation  III:  Rated  good  because  multiple 
sources  are  available. 

49.  Quantities/object/Standardization  I  II:  These  are  only  average 
with  single  sources.  However,  with  second  sourcing  they  share  the  advantages 
of  Accreditation  III. 

50.  Industry  participation/object/laissez  faire:  By  definition,  this 
strategy  permits  the  Program  Manager  to  design  his  computer  acquisition  to 
get  what  he  wants.  It  should  be  noted  that  this  does  not  necessarily 
maximize  competition. 

51.  Industry  participation/object/Accreditation  I:  This  is  similar  in 
business  prospects  to  other  single  developer  strategies,  but  is  generally 
with  lower  production  prospects. 

52.  Industry  participation/object/Accreditation  II:  The  supplier  foots 
the  whole  development  bill,  and  still  has  to  enter  a  Chinese  auction  to  get 
any  business. 

53.  Industry  participation/object/Accreditation  III:  The  supplier  must 
compete  at  least  twice  and  contribute  investment  ir,  development.  This  has  a 
slight  edge  over  Standardization  I  in  that  there  is  more  than  one  winner  of 
the  last  round. 

54.  Industry  participation/ object/Standardization  I:  Poorer  than 
Accreditation  III  because  there  is  only  one  winner  of  any  profitable  busines 
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55.  Industry  participation/ob j ect/Standardi zation  II:  This  more 
conventional  approach  limits  the  investment  required  for  a  contractor  to 
win  the  pot.  This  applies  also  to  second  sourcing. 

56.  Industry  participation/object/Standardization  I  with  second 

sourcing:  This  raises  industry's  willingness  to  participate  because  a 

loser  can  now  get  a  second  chance. 

57.  Technological  currency/object/laissez  faire:  Good,  because  con¬ 
current  development  is  always  an  option. 

58.  Technological  currency/object/Accreditation  I:  The  Program 
Manager  is  still  free  to  choose  the  latest  technology. 

59.  Technological  currency/object/Accreditation  II:  Rated  as  average 
because  sometimes  the  Program  Manager  will  have  to  take  an  option  that  is 
not  the  latest  vintage. 

60.  Technological  currency/object/Standardization  I  §  II:  A  central 
buy  holds  off  new  developments  until  justified  on  a  total  life  cycle  cost 
basis. 

61.  Hardware  development/source/laissez  faire  and  Accreditation  I: 

See  premise  8.  Sub-Panel  3  agreed  that  there  should  be  a  reduction  in  cost 
here  relative  to  the  object  level,  but  were  unable  to  agree  on  any  change 
in  the  overall  ranking  of  strategies  for  this  criterion. 

62.  Hardware  production,  hardware  logistics  8  quantity/source:  These 
are  identical  to  the  issues  at  the  object  level. 

63.  Software  development  and  software  logistics/source:  Admissibility 
of  more  architectures  increases  software  development  cost  for  all  categories 
except  laisscz  faire.  l.aissez  faire  remains  the  highest  in  cost,  but  the 
difference  is  no  longer  as  significant. 
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64.  Industry  participation/ source:  Commercial  VLSI  computers  can  be 
inexpensively  militarized,  and  this  fact  would  increase  participation  where 
it  was  otherwise  discouraged  by  heavy  investment.  The  sub-panel  was  unable 
to  agree  on  any  change  in  rankings. 

65.  Technological  currency/source:  Militarization  of  the  latest 
commercial  models  will  improve  currency  for  the  laissez  faire,  Accreditation  I, 
and  Accreditation  II  strategies.  These  three  had  already  been  rated  the  high¬ 
est  at  the  object  level. 

66.  Box  level:  The  sub-panel  observed  that  the  central  buy  strategies, 
Accreditation  III  and  the  two  Standardization  strategies,  really  assume  box 
level  standardization,  while  the  other  strategies  preclude  it.  Nonetheless, 
a  matrix  at  this  level  was  hurriedly  prepared  in  the  closing  minutes  of  the 
session,  and  is  presented  without  further  notes  of  justification,  save  the 
next. 

67.  Hardware  development,  hardware  production,  hardware  logistics,  and 
quantities/box:  Costs  improve  with  second  sourcing. 

4.7.6  Conclusions  -- 

1.  Central  buy  strategies  are  best  for  hardware  development,  production 
and  logistics  costs.  This  is  because  central  buy  strategies  do  not  permit  the 
development  of  a  new  computer  until  it  is  fully  justified  on  a  life  cycle  cost 
basis  versus  continued  use  of  its  predecessor (s) .  These  strategies  therefore 
minimize  the  frequency  of  computer  developments,  maximize  the  production  runs 
for  each  and  minimize  the  number  of  types  in  the  field. 

2.  Architectural  standardization  is  the  key  procurement  attribute  for 
moderating  software  costs.  By  permitting  the  development  of  mature  software 
production  enviroments,  it  has  a  strong  effect  on  moderating  software  develop¬ 
ment  costs;  and  considering  the  more  reasonable  requirements  for  maintenance 


.Ik 


— t-- .  r  "rfliynr'rfinin-'*'  ■  •  • 


-28- 


personnel  as  well  as  the  environmental  benefits,  it  has  a  very  strong 
effect  on  moderating  software  logistics  costs. 

3.  Sub-Panel  3  expressed  concern  that  the  cleanness  and  feasibility 
of  source  level  standardization  not  be  overstated.  True  IIOL  compatibility, 
as  defined  in  para  4.2,  cannot  be  achieved  without  stringent  controls  on 
machine  architecture  (word  length,  number  representation,  arithmetic  details, 
etc.)  Lifetime  maintenance  of  source  level  code  is  rare  and  hard  to  enforce. 
Assembly  level  code  will  continue  to  be  needed  --  as  a  minimum  for  1/0 
routines,  interrupt  handling,  resource  management,  and  diagnostics  --  and 
these  requirements  will  grow  as  code  is  written  to  control  distributed 
systems.  The  debugging  stage  of  software  development  continues  to  require 
knowledge  of  machine  architecture.  Thus,  standardizing  at  the  source  level 
falls  short  of  generating  many  of  the  advantages  of  ISA  standardization. 

In  any  case,  the  relative  ratings  at  the  source  level  are  much  the  same 
as  at  the  object  level  for  the  criteria  considered  by  the  sub-panel. 

4.  Multiple  sources  in  production  constitute  a  mixed  blessing  whose 
benefits  grow  with  production  volume.  Dividing  production  volumes  several 
ways  increases  costs,  but  competitive  pressures  reduce  them.  Commonality  at 
the  form,  fit,  function  and  interface  level  exacerbates  hardware  logistics 
costs;  while  commonality  at  the  build-to-print  level  leads  to  extra  non¬ 
recurring  hardware  costs  to  ensure  full  interchangeability,  a  set  of  problems 
with  which  industry  has  almost  no  successful  experience.  Advantages  of  assumed 
availability  are  of  secondary  importance. 

5.  As  the  Congress  insists  on  more  and  more  competitive  procurement 
strategies,  it  should  be  remembered  that  these  are  likely  to  be  less  and  less 
attractive  to  industry.  In  particular,  N  developers  -  1  producer  requires 
discouragingly  heavy  investment,  and  Accreditation  II  looks  unreasonably 
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risky  under  most  circumstances . 

6.  Although  the  less  restrictive  strategies  abet  the  attainment,  of 
technological  currency,  it  was  not  regarded  as  a  criterion  of  major 
importance. 

7.  The  sub-panel  felt  that  all  strategies  other  than  the  central  buy 
methods  really  preclude  box  level  standardization.  Note  that  this  view 
was  not  held  by  sub-Panel  1,  the  other  sub-panel  that  found  standardization 
level  to  be  a  factor.  They  felt  that  Accreditation  II  also  applied  to  the 
box  level. 

8.  As  VLSI  technologies  become  predominant  in  military  computers, 
the  criteria  for  selecting  procurement  strategies  will  change.  Hardware 
development  costs  will  generally  increase,  while  hardware  production  and 
logistics  costs  will  diminish  sharply.  Software  considerations  will  not 
generally  be  affected.  The  militarization  of  commercially  available  com¬ 
puters,  whose  chip  designs  and  fabrication  masks  are  in  place  and  whose 
software  development  environments  have  been  paid  for,  will  become  increas¬ 
ingly  attractive. 

9.  As  sub-Panel  3  worked  these  issues,  there  was  no  tendency  toward 
concensus  for  any  one  procurement  strategy  across  the  services. 
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4 . 8  General  Conclusions 

The  following  conclusions  are  in  addition  to  those  found  in 
paras.  4.5.6,  4.6.6  and  4.7.6. 

1.  Accreditation  relates  to  acquisition  strategies  rather  than  to 

an  extent  or  level  of  standardization.  Accreditation  includes  the  process 
of  certifying  that  a  product  meets  a  standard. 

2.  The  success  of  an  accreditation  strategy  will  be  driven  by  such 
factors  as 

a.  Hardware/software  standardization 

b.  Ability  to  satisfy  DoD  weapon  system  requirements 

c.  Acceptance  by  DoD  and  industry 

d.  Life  cycle  cost 

e.  Availability  of  qualified  personnel 

Taken  as  a  group,  the  two  standardization  practices  appeared  advantageous 
with  respect  to  more  evaluation  factors  than  the  group  consisting  of  the  three 
accreditation  strategies.  To  determine  the  optimum  strategy  for  a  given 
weapon  system  would  require  a  weighting  of  the  criteria  specific  to  that 
system. 

3.  Although  a  significant  data  base  of  issues  relating  to  acquisition 
strategies  for  standardization  was  addressed  and  evaluated  in  a  short  period 
of  time,  more  work  is  needed.  Analyses  conducted  in  depth  with  reference  to 
hard  data  resources  are  also  required.  For  example,  the  degree  of  industrial 
responsiveness  and  breadth  of  competition  under  each  of  the  acquisition 
strategies  deserves  further  quantitative  analysis. 

4.  There  has  not  been  sufficient  study  to  define  and  implement  a  single 
acquisition  policy.  Nevertheless,  the  potential  exists  for  common  embedded 
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computer  acquisition  strategies  across  all  four  services.  If  there  were 
a  single  acquisition  policy,  it  would  either  have  to  accommodate  both  central 
buy  strategies  and  strategies  employed  for  specific  weapons  systems,  or 
compromise  the  advantages  of  both. 
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RECOMMENDATIONS 


1.  The  panel  recommends  that  the  data  base  of  acquisition/accreditation 
issues  developed  at  Monterey  II  be  used  as  the  basis  for  further  study, 
leading  to  the  selection  of  computer  acquisition  strategies  best  able  to 
achieve  the  benefits  of  standardization. 

2.  Studies  of  computer  designs  Siould  be  conducted  to  determine  the 
impact  of  standardization  level.  These  designs  should  include  those  capable 
of  direct  HOL  execution.  The  level  of  detail  should  be  sufficient  to  fully 
expose  tie  implications  to  development,  life  cycle  cost,  and  support  of 
such  designs. 
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INTERFACE 


CONTROL  STRUCTURE 


INDUSTRY,  MILITARY  USERS 


ACCREDITATION  CONCEPT  -  GENESIS 


ASN (RE&S)  EMBEDDED  COMPUTER  REVIEW  PANEL 
PANEL  RECOMMENDATIONS  -  OCT  1978  REPORT 

1,  STRENGTHEN/TIGHTEN  MANAGEMENT  OF  COMPUTERS  IN  THE  NAVY 

2,  ACCELERATE  NEW  EMBEDDED  COMPUTER  SYSTEM 

3,  REGARDING  THE  UYK-7 

A.  DO  NOT  ADOPT  THE  RECOMMENDED  UYK-7  IMPROVEMENT. 

B.  INSTEAD,  ACCELERATE  THE  DEVELOPMENT  OF  THE  NEW 
NAVY  EMBEDDED  COMPUTER  (NECS) . 

C.  IF  NECS  IS  TOO  LATE,  DISTRIBUTE  ANY  OVERLOADED 
UYK-7  FUNCTIONS  TO  OTHER  7's  OR  20's. 

D.  IF  C  IS  NOT  FEASIBLE,  HOLD  A  COMPETITION  FOR  AN 
EMULATED  UYK-//UYK-7  UPGRADE. 

A .  ADOPT  A  POLICY  OF  MOVING  AWAY  FROM  TIGHT  STANDARDIZATION 

A.  TECHNOLOGY  WILL  YIELD  THIS  EVENTUALLY. 

B.  EXPEND  R/D  TO  ACCELERATE  THIS  MOVE. 


NECRP  REPORT 
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DEFINITION 


ACCREDITATION:  A  LIST  OF  ACCEPTABLE  COMPUTERS  IN  A 

PERFORMANCE  RANGE,  AS  OPPOSED  TO  ONE  COMPUTER  IN  A 
PERFORMANCE  RANGE,  WHICH  IS  STANDARDIZATION, 


NECRP  REPORT 


STANDARDIZATION 


POLICY  -  1  COMPUTER  FOR  A  GIVEN  PERFORMANCE 

RANGE 

REASON  -  OPERABILITY  ABOARD  SHIP 

OPERABILITY  -  RELIABILITY  +  MAINTENANCE 

AIDS  +  PEOPLE  +  PARTS 


NECRP  REPORT 


ACCREDITATION  STUDIES 


OBJECTIVE  -  OBTAIN  INDUSTRY  CONCEPTS  AND  ANALYSES 
IN  SUPPORT  OF  WORKABLE  ACCREDITATION 
APPROACH 

THEME  -  TO  PERMIT  THE  NAVY  TO  OBTAIN  THE  BENEFITS 
OF  NEW  TECHNOLOGY  AND  OPEN  COMPETITION  AS 
FREQUENTLY  AS  r  jSSIBLE  WHILE  AT  THE  SAME 
TIME  SATISFYING  OPERATIONAL  REQUIREMENTS, 
MILITARY  LOGISTICS  AND  COST  CONSTRAINTS 

SOURCES  -  FOUR  SEPARATE  STUDIES  AND  VIEWPOINTS 


AS  A  MINIMUM  THE  STUDY  SHALL  CONSIDER  THE  FOLLOWING  ISSUES 


o  REQUIREMENTS  FOR  OPERATIONAL  AVAILABILITY .  RELIABILITY, 
MAINTAINABILITY 

o  LOGISTICS  SUPPORT,  SPARES,  PERSONNEL,  TRAINING 

o  ENVIRONMENTAL  REQUIREMENTS 

o  OPERATIONAL  PERFORMANCE  REQUIREMENTS 

o  QUALIFICATION  AND  TESTING 

o  PERTINENT  TECHNOLOGY  TRENDS 

o  ABILITY  OF  RELIABILITY/MAINTAINABILITY  IMPROVEMENTS  TO 
DIMINISH  LOGISTICS  CONCERNS 

o  APPROACHES  TO  LOGISTICS  COMPATIBILITY  -  BOX  LEVEL  f3 , 
STANDARD  MODULES,  LEADER/ FOLLOWER  CONTRACTS,  ETC. 

o  COMMERCIAL  DEVELOPMENTS 

o  CRITERIA  FOR  ACCEPTANCE  AND  RETENTION  OF  COMPUTERS 

ON  ACCREDITED  LIST 

o  LIFE-CYCLE  COST  AS  ACCREDITATION  CRITERION 

o  FREQUENCY  OF  INTRODUCTION  OF  ACCREDITED  COMPUTERS  -  AT 
REGULAR  INTERVALS,  AS  REQUIREMENTS  DEVELOP,  ETC. 

o  PRACTICAL  LIMITS  ON  NUMBER  OF  VENDORS  AND/OR  COMPUTER 
TYPES 

o  LIMITS  OF  APPLICATION  OF  ACCREDITATION  CONTROLS  TO 
DIGITAL  PROCESSING  DEVICES 

o  ACQUISITION/ CONTRACTUAL  STRATEGIES  FOR  OBTAINING 
ACCREDITATION  CANDIDATES 

o  SOURCES  OF  FUNDING  FOR  DEVELOPMENT  AND  QUALIFICATION  - 
GOVERNMENT  OR  INDUSTRY 

o  PROMOTION  OF  COMPETITIVE  ENVIRONMENT 

o  TRANSITION  FROM  CURRENT  NAVY  PRACTICES  TO  ACCREDITATION 
--  APPROACH  AND  TIMING 


ISSUE  PRC  GIT 
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POLICY/ IMPLEMENTATION  EXTRACTIONS  FROM  STUDIES 

o  IMMEDIATELY  TO  FSED 

o  2  TO  4  FSED  FOR  PERFORMANCE  RANGE 

o  RE-DEVELOP  EVERY  "5"  YEARS  (WITH  CONSTANT  CHANGE  RATES) 

o  FVlSA  STANDARDIZATION  (ISA— >  HOL) 

o  2  OR  MORE  PRODUCERS  PER  PERFORMANCE  RANGE 

o  VISIBLE  LCC,  VENDOR  RO I /PAYBACK  &  EFFECTIVENESS  BENEFIT 
ANALYSIS  AGAINST  EXISTING  ACCREDITED  COMPUTERS 

o  INCLUDE  MICRO  &  SIGNAL  PROCESSOR  PRs 

o  INCREASED  NAVY  PRODUCT  MANAGEMENT 

o  VERTICAL  LINK  SYSTEM/PLATFORM/PROCESSOR  &  NAVY-WIDE 
LEVELS  WITH  LCC  &  MOW  ANALYSES 


UNRESOLVED  ISSUES 


o  ACCREDITED  COMPUTER  LINE 

-  STILL  HAVE  TO  RECOMPETE  FOR  EACH  BUY 

-  PIECEMEAL  BUYS 

-  NO  MARKET  CERTAINTY 

-  SPECIFICITY  OF  CRITERIA  FOR  LIMITING 
THE  N'TH  +  1  OFFER 

o  LOGISTICS  IMPACT 

-  MULTIPLE  SOURCES 

-  MAINTENANCE  DIFFICULTIES 

-  TRAINING 

-  SPARES 


Number  of  Vendors 


ACCREDITATION  PREREQUISITES 

o  NO  INCREASE  IN  LCC 

-  AND  - 

o  NO  INCREASE  IN  MANPOWER  NUMBERS  OR 
QUAL I F I  CAT I ON/TRA I N I NG  REQU I REMENTS 

-  AND  - 

c  NO  REDUCTION  IN  OPERATIONAL  AVAILABILITY 

-  AS  COMPARED  TO  STANDARDIZATION  - 


LOGISTICS 


o  LACK  OF  SPARE  PARTS  IS  SINGLE  LARGEST  CONTRIBUTOR 
TO  COMPUTER  DOWN  TIME  AT  SEA 

o  ACCREDITATION  WILL  REQUIRE  INCREASED  (OVER  STANDARD¬ 
IZATION)  ON-BOARD  SPARES  TO  MAINTAIN  GIVEN  LEVEL  OF 
OPERATIONAL  AVAILABILITY 

o  OPERATIONAL  AVAILABILITY  MUST  BE  MAINTAINED 

o  CONTINUED  SHORTAGES  OF  HIGH  GRADE,  HIGHLY  TRAINED 
COMPUTER/ELECTRONICS  MAINTENANCE  PERSONNEL  PROJECTED 
THROUGH  1986  AND  BEYOND 

o  ACCREDITATION  WILL  INCREASE  MAUPOWER/TRAINING 
REQUIREMENTS  FOR  SHIPBOARD  COMPUTER  MAINTENANCE 

o  NEW  R&M  TECHNOLOGY  AND  MILITARY  PERSONNEL  POLICIES 
WILL  BE  NEEDED  TO  OFFSET  THIS  BURDEN 

o  MORE  EXTENSIVE  USE  OF  STANDARD/ACCREDITED  PROCESSORS 
IN  PLACE  OF  NON-STANDARD  PROCESSORS  AND  SPECIAL  PURPOSE 
DIGITAL  ELECTRONICS  COULD  SIGNIFICANTLY  REDUCE  OVERALL 
SYSTEM  MAINTENANCE  TRAINING  AND  SPARING  REQUIREMENTS 


LIFE  CYCLE  COSTS 


o  ACCREDITATION  WILL  INCREASE  NON-RECURRING 

ACQUISITION  COSTS,  E.G.,  MULTIPLE  DEVELOPMENT, 
PRODUCTION  LEARNING  CURVES,  ETC. 

o  ACCREDITATION  WILL  INCREASE  MANPOWER 

TRAINING  AND  EQUIPMENT  LOGISTICS  COSTS 

o  ACCREDITATION  MUST  LOOK  TO  PRODUCTION  COMPETITION 
COST  BENEFITS  TO  OFFSET  OTHER  COST  BURDENS 

o  VALUE  (COST  PERCENTAGE)  OF  MARKET  COMPETITION 
NOT  DETERMINISTIC,  BUT  FIGURE  COULD  BE  "ESTAB¬ 
LISHED"  FOR  ACQUISITION  DECISION  PURPOSES 

o  "ESTABLISHED"  MARGINAL  VALUE  OF  COMPETITION 

CAN  PROVIDE  UPPER  LIMIT  CUT  OFF  POINT  FOR 
DETERMINING  NUMBER  OF  VENDORS  TO  ACCREDIT. 

LOW  BID  ON  COMPUTED  LCC  FORMS  BASELINE. 


SOFTWARE  COMPATIBILITY 


SOFTWARE  COMPATIBILITY  AMONG  ACCREDITED 
COMPUTERS  IN  A  PERFORMANCE  CUSS  IS  A 
NECESSITY 

NEAR  TERM  -  USE  THE  PROVEN,  ISA  COMPATIBILITY 
MECHANISM  WITH  STANDARD  HOLS 

LONG  TERM  -  MOVE  TO  THE  HOL  LEVEL 

-  DIRECT  HOL  MACHINE  (ISA  -  HOL)? 

-  IL  MACHINE  (ISA  =  IL)  WITH  STANDARD  HOL? 


POSSIBLE  ACCREDITATION  PROGRAM  OUTLINE 


o  ESTABLISH  MODEL  FOR  COMPUTING/COMPARING  LCC  OF 
CANDIDATE  ACCREDITED  COMPUTERS 

o  DEFINE  MINIMUM  ESSENTIAL  F3  SPECS  FOR  ACCREDITATION, 
i.e.,  PACKACE,  ENVIRONMENTAL.,  I/O  AND  SOFTWARE 
INTERFACES,  PERFORMANCE,  R&M ,  ETC. 

o  AGGREGATE  EMBEDDED  COMPUTER  REQUIREMENTS  PER  PERFORMANCE 
CLASS  FOR  3-5  YEARS  TO  ACHIEVE  ATTRACTIVE  MARKET 

o  SOLICIT  PROPOSALS  FROM  INDUSTRY  FOR  TWO  OR  MORE 
SUPPLIERS  PER  PERFORMANCE  CLASS 

o  OBTAIN  PRICE  QUOTES  FOR  VARIOUS  PRICE/ PRODUCTION 
RATES  AND  QUANTITIES 

o  CONTRACT  QUANTITIES/VENDORS  TO  MINIMIZE  JOINT  BUY  LCC 

o  LIMIT  JOINT-BUY  LCC  TO  FIXED  MARGIN  OVER  SINGLE 

LOWEST  BID  LCC.  THIS  WILL  LIMIT  NUMBER  OF  SUPPLIERS. 
SINGLE  SUPPLIER  IN  PERFORMANCE  CLASS  MAY  RESULT 
WHERE  MARKET  IS  TOO  SMALL  TO  SUPPORT  MULTI- VENDOR, 
SPLIT-MARKET  COMPETITION 

o  LIMIT  COMBINED  MAINTENANCE/TRAINING  AND  SPARING 
REQUIREMENTS  OF  NEW  ACCREDITED  COMPUTERS  TO  NOT 
GREATER  THAN  THAT  OF  THE  EXISTING  COMPUTER (S)  TO 
BE  DISPLACED.  THIS  WILL  ALSO  CAP  NUMBER  OF  SUPPLIERS. 

o  CONTRACT  FOR  VENDOR  PRODUCTION  RATE/QUANTITY 

o  REINITIATE  CYCLE  AT  3-7  YEAR  (5  YEAR?)  INTERVAL 

o  PROHIBIT/ DISCOURAGE  OIJT-OF-CYCLE  ACCREDITATIONS 

FROM  INDIVIDUAL  SOURCES,  EXCEPT  PERHAPS  FOR 
EXTRAORDINARY,  UNFORSEEN  TECHNOLOGICAL  BREAK¬ 
THROUGH.  MUST  STILL  SHOW  OVERALL  LCC  WIN  IN  FACE 
OF  MARKET  DISRUPTION.  TREAT  SPECIAL  REQUIREMENTS 
WITH  WAIVERS. 

o  REVIEW/UPDATE  ACCREDITATION  CRITERIA  PERIODICALLY 

o  KEEP  INDUSTRY  APPRISED 
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OBJECTIVE 


1 . 


Evaluate  existing  software  cost  estimating  model  s  a  nr! 
recommend  to  the  Joint  Logistics  Commanders  Joint  Policy 
Coordinating  Group  on  Computer  Resource  Management  a  Tri-Service 
approach  to  improve?  software  cost  estimating  methodology. 

2 .  SCOPE 


The  percentage  of  total  embedded  computer  systems  cost  that 
is  allocated  to  software  is  increasing.  Software  cost  estimating 
methodology  has  never  been  satisfactory,  and  it  is  becoming  more 
critical  that  a  satisfactory  methodology  be  developed.  Such  a 
methodology  should: 

•  Improve  the  ability  t.o  forecast  total  software  life  cycle- 
costs  during  the  Concept  Exploration  phase. 

0  Predict  softv/are  development  and  maintenance  costs  during 
the  Demonstration  and  Validation  phases  and  during  the  Full 
Scale  Development  phase. 

•  Allow  a  standard  for  evaluation  of  software  components  of 
pronosa 1 s . 

•  Provide  a  means  to  update  project  schedule  and  resource 
estimates . 

•  Incorporate  evolving  software  estimating  technology  and 
reflect  advances  of  improved  software  engineering 
technology . 

This  panel  was  asked  to  review  existing  software  estimating 
models,  and  assess  each  model’s  limitations  and  benefits, 
including  where  models  are  beneficial  and  where  they  are 
inadequate.  Further,  the  panel  was  to  establish  for  which  life 
cycle  phases  softv/are  costs  could  be  estimated,  and  assess  the 
accuracy  of  the  estimates  by  phase.  The  fundamentals  of  software 
cost  estimating  were  to  be  reviewed ,  and  an  assessment  made  of 
the  usefulness  of  parameters  and  the  desirability  of 
mul ti-varinble  algorithms. 

The  panel  was  also  asked  to  determine  software  cost 
estimating  needs  for  each  life  cycle  phase.  These  needs  were  to 
be  established  through  all  phases,  and  were  to  be  expressed  in 
terms  of  the  use  to  be  made  of  the  model,  the  output  desired,  the 
desired  accuracy  of  the  output,  and  the  input  required  to  drive 
the  model  .  The  means  of  using  the  methodology  were  to  be 
reviev/cd,  with  analyses  to  include  the  constraints  imposed  by 
resources,  and  the  probable  background  of  users.  The  contractual 
aspects  of  using  the  methodology  were  to  be  studied  and 
recommendations  for  implementing  the  methodology  developed. 
Finally,  a  software  cost  estimating  technology  program  needed  to 
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moot  r.oD  needs  was  to  bo  developed. 

3 .  APPROACH 
3 . 1  Preparation 

The  Panel  had  been  provided  references  (Appendix  C)  to  rend 
in  preparation  for  the  meeting,  prior  to  the  Vorkshop.  In 
addition,  Panel  members  brought  copies  of  reference  material  for 
use.  These  references  are  listed  in  Appendix  P. 

The  first  meeting  of  the  Panel  (see  Appendix  A  for 
attendees)  started  v/ith  presentations  on  individuals’  background, 
information  on  current  cost  models,  industrial  practices,  and 
automated  library  practices.  Copies  of  these  presentations  are 
provided  in  Appendix  D. 

3 • ?  I ssues 

Rased  upon  the  Panel  references  and  the  Panel  presentations, 
the  Panel  established  a  list  of  issues  which  are  tabulated  in 
Table  1.  The  items  in  this  table  express  concern  in  a  number  of 
different  ways.  The  Panel  concluded,  for  example,  that  most 
available  cost  models  are  deficient  in  some  way.  This  conclusion 
is  based  on  the  considerable  body  of  survey  information  already 
available,  and  it  was  concluded  that  additional  surveys  would  not 
be  productive.  There  also  '.'as  concern  that  existing  models  do 
not  meet  all  system  life  cycle  needs,  do  not  cover  all 
sof tware-rel ated  functions  (e.g.,  software  quality  assurance)  and 
require  considerable  expertise  to  use.  Finally,  there  was  a 
general  concern  about  legal  and  contractual  aspects  of  the  use  of 
models  in  the  acquisition  process. 

3.3  Issue  Consensus 


The  Panel  collectively  had  a  broad  background  and  the 
members  were  able  to  present  varying  viewpoints  on  the  subject, 
as  itemized  by  the  issues  in  Table  1.  A  list  of  items  on  which 
the  entire  Panel  could  concur  was  developed.  This  list  is  shown 
in  Table  ?.  A  significant  conclusion  is  that  DoD  has  common 
Software  Cost  Estimation  (SCE)  needs,  the  methodology  should  be 
directed  toward  embedded  weapons  systems  methodology  but  that 
this  SCE  methodology  was  not  necessarily  implemented  in  one  tool 
or  model.  It  was  suggested  to  the  Panel  that  the  DoD  automated 
data  processing  community  was  considerina  selecting  an  existing 
model  as  a  standard.  The  Panel  concurred  that  no  existing  RCE 
model  or  technology  would  satisfy  the  needs  of  the  embedded 
computer  system  community.  Therefore,  it  concluded  that, 
although  such  an  existing  methodology  or  model  might  be 
applicable  to  automated  data  processing  needs,  the  Panel  should 
restrict  its  attention  toward  an  RCE  methodology  that  is  directed 
toward  embedded  weapons  systems  nppl ient ions .  Diner  the 
methodology  would  be  at  the  DoD  level,  it  also  appears  that  a 
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central  organization  would  bo  required  to  control  the  process  and 
improve  it  as  technology  and  acquisition  regulations  change. 

The  Panel  failed  to  concur  on  only  one  point:  should  cost 
data  reporting  be  tied  to  present  financial  reporting.  This 
could  have  legal  implications  about  v/hicli  the  Panel  could  not 
achieve  a  consensus  without  considerable  nnoly?  s. 

3 . 4  Issue  Resolution 

The  issues  and  consensus  items  (Tables  1  and  2)  here  divided 
into  the  three  principal  groups  indicated.  Three  Pub-Panels  were 
formed,  and  items  assigned  to  the  three  Sub-Pariel  s  for 
development  of  appropriate  conclusion!,.  The  Sub-Panels 
addressed:  (1)  definition  of  the  SCF,  methodology  requirements 
for  all  aspects  of  the  system  life  cycle  rind  the  expected  use  of 
the  methodology;  (2)  evaluation  of  both  the  existing 
methodologies  and  the  Panel's  derived  requirements,  and 
recommendations  for  research  and  development  of  the  required 
methodology;  and  (3)  legal  and  contractual  implications 
associated  with  using  a  DoD  -  level  SCH  methodology.  The 
Sub-Panel  findings  are  provided  in  Section  4 .  Members  of  the 
three  subpanels  were: 


Sub-Panel  1 
Life  Cycle 
Requirements 


Robert  Earnest 
Ronald  Leask 
Herman  Oberkrom 
Richard  Page 
David  Thornell 
David  Usechak 


Sub-Panel  2 
Model  Technoloc 


Roger  Bate 
Deane  Bergstrom 
Joseph  Duquette 
James  McCall 
Robert  Paulsen 


Sub-Panel  3 
Contractual 
Considerations 

Marilyn  Fujii 
Clell  Gladson 
Richard  Latshaw 
George  Robertson 
Newnam  Thompson 
George  Trever 


The  panel  co-chairmen,  Dean  Hartwick  and  Robert  Berri, 
coordinated  work  among  the  Sub-Panels.  For  the  remainder  of  the 
workshop,  the  Panel  primarily  worked  within  the  subpanel 
sessions,  with  periodic  meetings  to  prepare  the  presentations  to 
the  entire  workshop. 


4.  DISCUSSION 

Many  different  software  cost  estimating  models  have  been 
developed  and  used  by  DoD  and  industry,  with  varying  results. 
Within  DoD,  SCE  models  have  most  commonly  been  used  to  develop 
independent  cost  estimates  for  budgetary  purposes.  More 
recently,  they  have  been  used  to  evaluate  sources  and  support 
software  developer  selection. 

Most  DoD  contractors  use  software  cost  estimation  mc»dels 
during  proposal  activities.  In  some  cases  these  models  are 


proprietary  and  have  boon  developed  solely  by  the  contractor 
organization  based  on  its  past  deve]  opment  experiences.  However, 
because  of  the  use  by  government  organizations  of  one  of  the 
commonly  available  cost  models  and  because  the?  development  of  an 
accurate  cost  model  is  difficult,  most,  contractors  tend  to  use 
one  or  more  of  the  common  models.  The  use  of  software  cost 
estimation  mcxTbls  by  contractors  is  almost  always  a  supplementary 
activity  to  the  traditional  bottoms- up  engineering  estimate 
generated  by  the  technical  contributors  to  the  proposal.  The 
bottoms-up  engineering  estimate  can  be  very  accurate  if  the 
development  is  bcased  on  similar  past  experiences  of  the  technical 
personnel.  The  disadvantages  of  this  technique  for  JLC  use  is 
its  questionable  applicability  to  new  or  unique  developments  and 
the  requirement  that  a  significant  amount  of  preliminary  design 
effort  is  required  to  effectively  maV.e  a  bottoms-up  engineering 
estimate.  This  detail  of  information  is  not  generally  available 
nor  is  the  time  or  resources  available  to  generate  it  early  in 
the  acquisition  life  cycle  when  estimates  are  required. 

The  organizations,  both  government  and  industry,  that  are 


making  effective 

characteristics  in 
characteristics  are: 

use 
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the*  models 
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certain 
model  s  . 

common 
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•  The  models 

are 

used 

v  i.  th  i  n  a 

cost 

estimation 

and 

evaluation  methodology  which  tightly  controls  the  input  to 
and  use  of  the*  models.  These  methodologies  cause  a  certain 
discipline  to  be  enforced  and  result  in  consistent, 
repeatable,  and  documented  use  of  the  models. 

•  The  models  have  been  calibrated  and  tuned  to  the  specific 
application  environment  in  which  they  are  used. 

•  The  models  are  used  to  perform  risk  analysis  and  identify 
sensitivities  and  deficiencies. 

•  The  models  only  supplement  common  sense  and  the  estimates 
of  knowledgeable  and  experienced  personnel. 

•  The  methodologies  incorporate  a  concept  of  evolutionary 
development  and  refinement  of  the  cost  estimation  models. 

In  Section  4.1,  the  SCE  models  in  common  use  by  DoD  and 
industry  are  summarized  according  to  modeling  parameters, 
applicability  to  different  life  cycle  phases,  and  how  closely  the 
models  estimate  actual  efforts.  A  series  of  conclusions  are 
drawn  in  Section  4.1,  but  an  overall  assessment  is  that  no 
existing  SCE  model  or  methodology  can  even  remotely  be  considered 
now  as  a  basis  for  a  JLC  embedded  computer  software  cost 
estimating  standard. 


One  of  the  characteristic  shortcomings  attributed  to  SCE 
models  is  that  they  have  all  been  developed  to  satisfy  an 
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Table  3.  Summary  of  SCE  Model  Surveys 
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exiFting  software  development  environment.  In  faction  4.2,  the 
requirements  that  fCE  methodology  (including  the  use  of  rCE 
models)  must  satisfy  to  adequately  estimate  DoD  embedded  computer 
software  life  cycle  costs  are  discussed.  Three  different  needs 
are  formulated  -  using  the  fCE  methodology  to  prepare  long  range 
budget  estimates?  supporting  software  acquisition;  and  to 
support  software  maintenance  and  enhancement  after  deployment. 
Additionally,  the  necessity  of  the  f.CE  methodology  to  react  to 
improved  software  engineering  technology  and  to  satisfy 
operational  needs  are  considered. 

Ml  SCE  models  gain  their  credibility  and  responsiveness  as 
a  result  of  the  data  used  in  tbeir  development.  A  shortcoming  of 
most  models  is  that  a  coherent  and  we) 1-def ined  data  base  has  not 
been  consistently  collected  or  maintained.  In  Section  4.3,  this 
problem  is  discussed,  and  an  approach  that  the  JLC  can  follow  to 
correct  these  problems  is  proposed.  This  approach  consists  of 
establishing  p  single  Government  agency  to  serve  as  an  ECE  data 
repository,  and  assigning  it  the  task  of  defining  data  collection 
techniques  and  procedures.  Also  it  is  concluded  that  MIL-HTD 
HP.1A  should  be  modified  to  better  reflect  the  complexity  of 
computer  software. 

A  most  important,  consideration  in  the  use  of  fCE  methodology 
is  the  manner  in  which  it  should  be  used.  In  flection  4.4,  the 
ramifications  of  using  CCE  model  results  are  considered.  Among 
the  conclusions  drawn  there  are  that  GCE  methodology  should  not 
be  used  to  determine  foes  and  progress  payments.  However,  KCE 
methodology  can  be  used  to  support  source  selection  and  to  aid  in 
Government  technical  performance  measurement. 

A  final  area  of  discussion,  Section  4.5,  relates  to  the 
question  of  the  evolution  of  SCE  methodology.  That  is,  given  the 
present  stato-of-art  and  the  obvious  needs  of  the  embedded 
computer  software  community,  what  steps  can  be  taken  by  JLC  to 
encourage  technology  advancement.  It  is  concluded  that 
procedures  and  guidelines  should  be  established  to  support  DoD  in 
using  existing  SCE  methodology  over  the  next  three  years,  while 
JLC  should  sponsor  a  technology  improvement  program  to  achieve  a 
common  methodology  in  the  next  three  to  seven  years. 

4 . 1  State-of-the-Art  Of  Software  Cost  Estimation  Technology 

Several  extensive  surveys  of  cost  models  have  been  conducted 
under  DoD  sponsorship  to  assess  the  utility  of  current  software 
cost  estimation  models.  Table  3  identifies  six  surveys  and  the 
models  evaluated  during  those  surveys.  Also  indicated  in  Table  3 
are  the  assessment  techniques  utilized  in  conducting  the 
evaluation.  These  assessment  techniques  include  evaluating  the 
features  of  the  model ,  what  cost  factors  are  taken  into  account 
in  the  model,  what  life  cycle  phases  are  covered  by  the  model, 
and  how  accurately  the  model  predicted  the  costs  of  actual 
software  developments. 


These  surveys  represent  evaluation  of  the  model s  from  the 
following  application  viewpoints: 

•  Management  information  systems 

•  Command  and  control 

•  Data  base  management  systems  -  commercial 

•  Command,  control,  communication  and  intelligence 

•  Flight  dynamics 

•  Real-time  applications 

•  Space,  aircraft,  avionics 

•  Weapon  systems 

•  Scientific  and  engineering 

•  Logistics  and  maintenance 

In  each  case  a  structured  consistent  evaluation  process  was 
utilized.  To  illustrate  some  of  the  findings: 

e  Table  A  identifies  what  cost  factors  are  accounted  for  in 
each  of  the  cost  models  surveyed.  Note  that  no  one  model 
considers  all  of  the  factors  that  intuitively  have  an  impact 
on  the  cost  of  a  software  development  project. 

•  Table  5  identifies  what  cost  elements  and  what  life  cycle 
phases  arc  covered  by  each  of  the  cost  models  surveyed. 
Table  6  represents  the  same  evaluation  from  another  survey. 
In  this  cose  a  weight  or  score  was  assigned.  Note  that  the 
phase  coverage  of  the  models  is  limited.  Either  the  model 
does  not  cover  the  phase  or  does  not  cover  it  in  the  level 
of  detail  required. 

•  Table  7  provides  an  error  evaluation  of  several  of  the 
models  across  three  very  distinct  application  areas:  a 
commercial  data  base  management  system,  the  Air  Force  Data 
System  Design  Center,  and  NASA/Goddard  flight  dynamic 
systems.  Note  that  the  models,  in  general,  demonstrated 
poor  accuracy  in  estimating  actual  costs  and  also  their 
accuracy  varied  greatly  from  one  appl ication  environment  to 
another . 

•  Table  h  describes  the  results  of  an  accuracy  assessment 
within  one  development  environment  for  one  application.  The 
findings  here  indicated  that  a  very  simple  model,  the  SEL 
model,  developed  strictly  from  local  data,  was  as  effective 
as  more  complex  and  expensive  models. 
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TABLE  4.  COST  FACTORS  CONSIDERED  IN  COST  ESTIMATION  TECHNIQUE 


COST  FACTORS 

OOTY 

IBM 

GRC 

PUTNAM 

SDC 

TIM 

PRICE-S 

OOEINO 

i 

DEVELOPMENT  EXPERTISE 

j 

PERSONNEL  EXPERIENCE  ANO 

QUALIFICATIONS 

X 

l 

X 

1 

X 

i 

DESIGN  PERSONNEL  PARTICIPATION  IN  ' 
DEVELOPMENT 

X 

X 

■ 

EXPERIENCE  WITH  COMPUTER  SYSTEM 

X 

1 

1 

EXPERIENCE  WITH  PR0MW*ING  LANGUAGE 

X 

X 

X 

I 

i 

EXPERIENCE  WITH  APPLICATION 

X 

X 

1 

X 

i 

CUSTOMER  ENVIRONMENT 

CUSTOMER  INTERFACE  COMPL'XITY 

X 

X 

j 

USER  participation  in  reouircments 

OEFINITION 

X 

j 

REOUIRCMENTS  STABILITY 

CUSTOMER  EXPERIENCE  WITH  APPLICATION 

X 

X 

X 

X 

X 

I 

X 

1 

SECURITY  REQUIREMENTS 

DOCUMENTATION  REOUIRCMENTS 

MULTIPLE  OPERATIONAL  SITES 

MULTIPLE  CONTRACTORS 

X 

X 

X 

X 

I 

X 

i 

•3 

=>1 

1 

] 

s 

FORMALIZED  TOC 

1 

1 

TRAINING  OF  OPERATING  AGENCY 

-I 

i 

5 

'J 


COST  FACTORS 

OOTY 

IBM 

GRC 

PUTNAM 

— 

SDC 

TIM 

PRICE-S 

OOEIHO 

DEVELOPMENT  ENVIRONMENT 

WINDER  OF  PERSONNEL 

OURAT ION  OF  PERSONAL  ASSIGNMENTS 

X 

X 

X 

CONCURRENT  HARDWARE/ SOFTWARE 
DEVELOPMENT 

X 

X 

X 

I 

ACCESS  TO  OEYELOPUV  COMPUTER 

X 

X 

X 

USE  OF  MOOERN  PROGRAMMING  PRACTICES 
OETAIL  OF  REQUIREMENTS  OEFINITION 

X 

X 

X 

X 

1 

DEVELOPMENT  VIA  INTERACTIVE  OR  BATCH 

X 

X 

X 

X 

X 

DEVELOPMENT  AT  OPERATIONAL  SITE 

X 

X 

I 

OPERATIONAL  SYSTEM  DIFFERENT  FROM 

development  system 

X 

X 

DEVELOPMENT  AT  MORE  THAN  ONE 

LOCATION 

X 

X 

CAPABILITY  OF  TOOLS  ANO  AIDS 

SUPPORT  PERSONNEL  AVAILABILITY 

RELIABILITY  OF  DEVELOPMENT  SYSTEM 

MANAGEMENT  TCCIMIQUES 

STANDARDS  ANO  PROCEDURES 

CONFIGURATION  MANAGEMENT 

X 

1 

X 

IMPLEMENTATION  LANGUAGE  HOt/MOL 

rwirm 

•X 

X 

X 

X  -  MOOCI.  DIRECTLY  CONSIDERS  FACTOR 
I  -  MODEL  INDIRECTLY  CONSIDERS  FACTOR 


Reference: 

Junk,  et  al,  "Survey  of  Software  Cost 
Estimatina  Techniques,"  GE/MDSO 
CIS  010,  May  1978 
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Table  4.  Cost  Factors  Considered  in  Cost  Estimation  Technique  (Can't.) 


X  -  MOOEL  DIRECTLY  CONSIDERS  FACTOR  1  SECONDARY  RELATIONSHIP 

I  -  MOOEL  INOIRECTLY  CONSIDERS  FACTOR 
P  •  MOOEL  PARTIALLY  CONSIDERS  FACTORS 
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TABLE  5 


EVALUATION  OF  COST 


TABLE  6.  SUMMARY  OP  MODEL  COMPLIANCE 


WITH  AIR  FORCE  ESTIMATING  REQUIREMENTS 


ESTIMATING  SITUATION 


H 


3 

§ 


w 

g 

< 

z 

o 

PS 

N 

W 

o 

< 

u 

to 

Eh 

& 

o 

H 

«* 

5 

PS 

to 

z 

a 

w 

•4 

pi 

o 

H 

u 

S 

O 

> 

PS 

W 

Q  Eh 

(Xj 

H 

H 

O 

►4 

O 

9  2 

Jflj 

PS 

►4 

W 

o 

< 

ca 

2  2 

fn 

p. 

to 

Eh 

s 

1 .  CONCEPTUAL 

Scope 

Detail 

2 .  CONCEPTUAL 

Scope 

Detail 

3 .  CONCEPTUAL 

Scope 

Detail 

4 .  VALIDATION 

Scope 

Detail 

5.  FULL  SCALE  DEVELOPMENT 

Scope 

Detail 


244113544 

555445555 

24411354  4 

444555444 

344224544 

243224334 

334224534 

132113214 

334334534 
1321132  14 


NOTE:  Numbers  indicate  degree  to  which  the  model  satisfies 
the  particular  estimating  requirements.  5  indicates 
nominal  satisfaction  of  the  requirement. 


Reference:  Thibodeau,  "An  Evaluation  of  Software 
Cost  Estimating  Models'*,  RADC  TR-,  February,  1981 
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TABLE  7.  SUMMARY  OF  MODEL  ESTIMATING  PERFORMANCE 


RMS  ERROR* 


MEAN  PROJECT  SIZE 


D  A 

T  A  S 

E  T 

MODEL  TYPE 

COMMERCIAL 

DSDC 

SEL 

REGRESSION 

A  AEROSPACE 

1.35 

2.11 

0.605 

B  DOTY 

1.05 

C  FARR  &  ZAGORSKI 

16.9 

D  TECOLOTE 

4.36 

/ 

E  (Recalibrated) 

0.643 

0.933 

0.309 

HEURISTIC 

F  BCS 

0.787 

G  DoD  MICRO 

1.26 

H  PRICE-S 

0.383 

1.44 

0.297 

I  TRW/WOLVERTON 

0.927 

PHENOMENOLOGICAL 

J  SLIM 

0.246 

0.216 

0.865  ' 

*RMS  ERROR  -[  j  £  <ACTi  _ESTi,  2]‘= 

t-1 


Reference:  Thibodeau,  "An  Evaluation  of  Software  Cost 
Estimating  Models,"  RADC  TR-,  February  1981. 


TABLE  8.  AVERAGE  %  ERROR  FINDINGS 


MODELS  AVE  %  ERROR* 


DOTY 

80 

WALSTON-FELIX 

16 

TECOLOTE 

19 

GRC  AGG 

112 

SLIM 

19 

PRICE-S 

20 

SEC 

16 

*  Applied  to  8  actual  projects. 

Reference;  Cook,  "An  Appraisal  of  Selected 
Models  for  Software  Systems",  NASA  X-582-81-1 
December  1980. 
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•  Figure  1  illustrates  a  comparative  evaluation  of  the 
effort-time  trade-off  between  three  models.  The  Putnam 
model  and  PRICE-3  model  are  the  only  models  which  recognize 
this  tradc-off. 

The  conclusions  drav.n  from  these  surveys  are  as  follows: 

Conclusion  1  ;  The  statistical  confidence  with  which  one  can 
use  the  current  cost  estimation  models  is  quite  low.  The 
demonstrated  accuracy  of  current  cost  estimation  models  is 
insufficient  for  JLC  application. 

Conclusion  2:  The  current  cost  estimation  models' 
performance  varies  greatly  from  one  development  environment 
to  another  and  from  one  application  to  another. 

Conclusion  3:  The  current  cost  estimation  models  do  not 
typically  cover  all  of  the  life  cycle  phases  of  a  software 
product  in  the  required  level  of  detail. 

Conclusion  4:  The  current  cost  estimation  models  do  not 
typically  cover  all  of  the  cost  elements  in  the  required 
level  of  detail  nor  are  they  related  to  the  1,'or k  Breakdown 
Structure  of  the  development. 

Conclusion  5:  Complicated  models  have  not  proven  to  perform 
better  than  very  simple  models. 

Conclusion  6:  The  burden  of  an  accurate  estimate  is  on  the 
user.  The  current  models  require  full  understanding  of 
their  characteristic  behavior  to  a  particular  environment 
even  though  this  behavior  cannot  be  logically  related  within 
the  model  to  specific  characteristics  of  the  development 
environment.-  The  user  is  also  required  to  fully  understand 
the  definition  of  the  input  parameters  even  though  user 
documentation  for  the  models  is  typically  poor.  In  order  to 
minimize  estimation  error  the  user  must  do  extensive 
calibration  and  tuning  of  the  models. 

Conclusion  7:  Current  cost  estimation  models  are  better 
able  to  satisfy  needs  of  JLC  early  in  acquisition  life 
cycle . 

Conclusion  0:  No  one  current  cost  estimation  model 
satisfies  all  of  the  JLC  cost  estimation  needs. 

Conclusion  9:  Reliable  historical  data  for  model 
development  or  validation  is  almost  non-existent. 

The  following  technology  needs  exist  based  on  this 
assessment  of  the  present  state-of-the-art  of  software  cost 
estimation  models; 
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Conclusion  10:  A  software*  cost,  estimation  (HCF. )  methodology 
( incl ud ing  definition  of  PCE  models)  should  be  developed 
which  covers  the  life  phases  completely  by  providing  the 
required  cost  estimation  information  to  the  user  based  on 
data  available  at  ..hat  time  in  the  acquisition  life  cycle. 
They  should  be  adaptable  to  development  environment, 
application,  and  software  development  methodology,  and  be 
capable  of  being  automated,  with  tools  to  support  decision 
analysis  . 

Conclusion  1 1 :  Basic  research  should  be  conducted  into 
techniques,  determining  parameters  that  characterize  the 
software  development  environment,  and  the  influence  of 
application  environment  upon  cost  model  accuracy. 

Conclusion  12:  \7hile  no  more  surveys  appear  to  be  required, 
additional  evaluation  of  why  the  current  models  perform 
differently  in  different  application  and  development, 
environments  would  provide  additional  insight. 

4 . 2  DoD  Software  Estimating  Requirements 

4.2.1  Life  Cycle  Considerations 

Software  cost  estimates  are  typically  of  two  classes: 
software  lifecycle  costs  for  a  specific  system,  and  cost  and 
schedule  for  a  specific  change  to  the  software  of  the  specific 
system.  prior  to  developing  recommendations  for  an  overall  FCE 
methodology,  the  issue  of  v/hat  FCEs  are  required  at  various 
phases  in  the  system  life  cycle  (Figure  ?)  in  order  to  suoport 
budget  actions  and  management  program  planning  must  be  addressed. 
Within  the  various  phases  of  the  software  system  life  cycle 
certain  types  of  information  are  needed  to  provide  costing 
estimates.  These  estimates  can  then  be  used  for  system  life 
ycle  cost  estimating  and  budgeting. 

An  SCE  methodology  can  be  used  in  Concept  Exploration  to 
perform  cost/risl:  assessments.  This  is  the  first  opportunity  to 
examine  longer  range  costs  of  software,  given  a  reasonable, 
though  imprecise  system  concept  together  with  gross  functional 
allocations  of  hardware/ software .  The  SCE  methodology,  used  with 
a  historical  data  base  of  cost  information.  can  be  used  to 
formulate  rough  estimates  of  system  cost  and  potential  risK  in 
development.  A  model  is  needed  to  generate  the  above  output 
during  the  early  phases  of  the  system  life  cycle.  Different 
types  of  data  are  required  for  this  early  process.  First  a  need 
or  operational  requirement  should  have  been  stated  which  as  a 
minimum  should  contain  system  performance  testing  requirements, 
and  support  philosophy  (i.e.  maintenance  and  manning). 

Secondly,  the  type  of  (hardware  and  software)  technology 
anticipated  for  use  on  the  program  must  be  included.  Third, 
historical  system  data  derived  from  past  similar  systems, 
technologies,  and  methodologies  should  be  included  to  form  the 
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Figure  2.  System  Acquisition  Life  Cycle 


initial  cost  estimating  base.  Figvire  3  is  a  graphic 
representation  of  this.  At  the  end  of  any  of  the  software  system 
life  cycle  phases  and  before  the  start  of  the  next  phase,  more 
refined  cost  estimate  data  are  required  in  order  to  allow  for 
appropriate  decisions. 

In  the  Demonstration  and  Validation  phase,  SCE  methodology 
can  be  used  to  assist  the  Government/Contractor  in  estimating  the 
cost  of  a  particular  system  design  on  either  a  near-term 
development  basis  or  a  longer  term  life  cycle  cost  basis. 
Variations  of  the  design  can  also  be  tested  for  cost  sensitivity 
and  allocations  of  functions  can  be  shifted  from  across  the 
hardware/ software  boundary  to  reach  an  acceptable  system  cost 
advantage  or  cost/benefit  tradeoff.  As  a  new  technology  or 
design  approach  is  considered,  ECE  methodology  can  assess  the 
cost  advantage  of  a  new  "strawman"  design  in  comparison  with 
earlier  technology.  The  types  of  input  and  output  data  required 
for  this  detailed  estimation  are  shown  in  Figure  4. 

As  a  system  fol 1 ows  the  acquisition  cycle  as  given  in  Figure 
2,  a  more  detailed  cost  estimate  is  needed  very  early  in  the 
Full-Scale  Development  phase.  As  the  decision  is  reached  to 
build  a  particular  item  in  the  Full  Scale  Development  phase,  more 
accurate  assessments  of  cost/performance  can  be  made  through 
hardware/ software  allocations  and  tradeoff  analyses.  The  basic 
design  is  unfolding  and  the  results  of  analyses  become  more 
precise  and  firm.  The  hardware/software  design  is  more  visible, 
and  historical  data  being  acquired  during  the  course  of  the 
contract  are  more  exact.  However,  the  same  methodology  shown  by 
Figure  4  can  be  used,  merely  using  the  better  input  to  improve 
estimates .  The  SCE  methodology  can  expose  preferred  design 
approaches  as  various  parts  of  the  system  are  more  finely  tuned. 
At  this  point,  the  most  cost-effective  approach  should  appear  in 
terms  of  development  potential. 

During  the  production  phase,  SCE  methodology  can  be  used  to 
assess  all  cost  impacts  due  to  the  many  late  changes  that  must  be 
incorporated  into  the  production  design.  Incorporating  multiple 
software  changes  into  the  production  version  of  the  system  may 
become  increasingly  complex  because  of  the  necessity  to  integrate 
modified  CPCIs  into  a  finished  software  package.  Here,  SCE 
methodology  assists  the  Government/Contractor  in  assessing  the 
overall  cost  impact  of  late,  and  possibly  voluminous,  changes  to 
production  versions  of  system  software.  During  software 
maintenance,  especially  for  large-scale  ground  based  systems 
deployed  in  widely  dispersed  geographic  locations,  extensive 
support  resources  can  be  required  to  maintain  system  software. 
SCE  methodology  should  be  applied  to  determine  the  long  term  cost 
of  support  services.  Because  of  the  eriticol  requirement  for 
weapon  system  readiness,  adequate  staffing  with  skilled  personnel 
implies  a  costly  continuing  training  and  retraining  activity  to 
be  carried  on  throughout  the  system  life  cycle. 
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Figure  4.  SCE  Parameters  During  Demonstration  and  Validation  Phase 


As  soon  as  configuration  updates  are  required  to  bane) inrd 
software,  there  is  a  need  for  management  to  accurately  estimate 
the  cost  and  schedule  associated  with  each  oarticulor  update  as 
well  as  the  impact  that  the  update  nay  have  on  the  software  life* 
cycle  cost.  The  methodology  that  is  needed  to  fulfill  that 
requirement  is  shown  in  Figure  5.  The  methodology  would  have  as 
input  data  uniquely  associated  with  the  software  update  in 
question.  The  specific  character ics  of  the  software  would  be 
quantifiable  in  detail,  as  would  the  testing  requirements,  the 
capabilities  of  the  testing  facilities  and  tools,  as  well  as  the 
qualifications  of  the  personnel  involved  in  the  update  process. 
As  DcD  systems  become  increasingly  integrated  with  each  other, 
the  methodology  must  account  for  the  influence  of  systems 
interoperability  requirements  as  well  as  mandated  schedules  for 
the  completion  of  the  update.  The  output  should  be  an  update 
cost  and  a  schedule  which  is  apportioned  into  the  functions  or 
phases  as  shown.  Separate  local  user  adaptation  of  the 
methodology  should  be  authorized  to  allow  the  system  program 
management  personnel  to  develop  cost  and  schedule  estimates  for 
elements  or  subsets  of  the  total  system  update  for  internal 
planning  and  management  purposes. 

Conclusion  13:  Based  upon  an  evaluation  of  the  intended  use 
of  the  SCE,  and  the  available  input  data  during  the  system 
life  cycle,  it  is  determined  that  three  classes  of  software 
cost  estimating  methodology  are  needed.  RCE  methodology  is 
needed  during  the  Concept  Exploration  phase  when  input  data 
are  limited  to  historical  life  cycle  data  on  comparable 
systems.  The  second  level  of  RCE  methodology  is  an 
enhancement/refinement  of  the  first  in  that  it  incorporates 
estimates  based  upon  specific  design  characteristics  of  the 
system  as  they  evolve  during  the  development  process.  The 
primary  output  is  an  estimated  system/ so ftware  life  cycle 
cost.  The  last  level  of  SCE  methodology  uses  actual 
historical  data  on  the  specific  system  and  is  used  to 
estimate  cost  and  schedule  associated  with  specific  software 
updates.  All  of  the  software  cost  estimating  methodologies 
should  be  usable  at  the  location  of  the  organization  having 
management  responsibility  for  the  software. 

4.2.2  DoD  Goals 

A  major  element  of  the  DoD  SCE  goals  is  establishing  a 
reasonable,  representative,  and  standardized  SCE  methodology. 
This  is  not  to  say  that  DoD  should  select  a  specific  model  of  the 
current  state-of-the-art  and  declare  it  a  standard.  The  Panel 
concluded  that  there  are  many  dangers  in  doing  this.  The  Panel 
believes  that  DoD  would  be  better  advised  to  specify  the  general 
procedure  for  estimating  software  costs  (i.e.,  major  activities 
model  selection,  model  documentation,  estimate  documentation  and 
management  actions  required  to  use  the  results  of  any  software 
cost  estimating  effort) .  The  establishment  of  this  estimating 
methodology  should  be  in  concert  with  the  data  collection  goals 
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and  should  make  use  of  the  data  collected  to  "fine-tune"  current 
models  and  develop  new  models.  The  model /methodology  development 
should  possess  the  following  attributes. 

•  Open  discipline  -  A  procedural  envelope  could  be  defined 
in  which  rationale  for  specific  model  selection  and  exercise 
is  provided.  This  allows  flexibility  in  estimating  while 
providing  a  disciplined  approach  to  the  development  of  a 
software  cost  estimate. 

•  The  use  of  multiple  models  -  It  should  be  pointed  out  that 
the  term  methodology  is  meant  to  imply  the  approach  that  is 
taken  to  employ  specific  FCE  models.  This  methodology 
should  allow  for  the  employment  of  a  number  of  models  as 
required  by  life  cycle  phase  or  for  application.  This 
attribute  will  allow  for  a  best-fit  situation  and  yield  the 
most  accurate  results. 

•  Reproducible  -  Given  the  sane  situation  and  the  same  data, 
two  independent  activities  should  be  able  to  produce  the 
same  estimate  of  cost.  This  attribute  provides  for  accuracy 
assessments  and  auditability  in  the  software  cost  estimating 
exercise.  In  providing  such  an  attribute,  comparative 
trade-off  studies  are  enhanced,  source  evaluation  is 
expedited  and  continuity  in  program  monitoring  activities  is 
achieved . 

•  Living  Methodology  -  ’.Ihatever  methodology  is  chosen,  it 
must  constantly  be  updated  to  reflect  the  current  state  of 
software  technology.  This  attribute  is  achieved  though 
institutionalizing  methodology,  and  through  DoD 
instructions,  manuals,  regulations  and  standards.  These 
should  be  constantly  updated  to  reflect  the  results  of 
current  research  in  the  SCE  activity. 

Another  DoD  FCE  goal  should  be  the  encouragement  and  support 
of  software  cost  estimating  research.  A  comprehensive  DoD  SCE 
technology  development  plan  should  be  developed,  identifying  FCE 
research  requirements  into  the  future.  The  DOD  centralized  group 
should  be  charged  with  the  management  of  this  plan  and  be 
provided  with  sufficient  resources  to  accomplish  the  objectives 
of  the  plan.  Clearly  the  SCE  technology  research  should  address 
itself  to  near  term,  interim,  and  long  term  objectives,  all  of 
which  support  the  data  collection  and  model  development 
activities . 

There  are  certain  general  characteristics  which  SCE  models 
should  have  in  order  to  be  useful  and  cost-effective  in  the 
variety  of  applications  expected  of  them.  Some  of  these  are 
listed  below.  Special  mention  should  be  made  of  the  general 
finding  of  surveys  of  present  SCE  models  that  they  are  sensitive 
to  the  development  environment.  It  is  clear  that,  for  management 
of  development  programs,  models  that  contain  adjustable 
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parameters  to  reflect  different  environments  are  appropriate,  but 
there  is  a  need  for  models  that  compute  the  "should-cost"  amount 
for  the  best  possible  environment  in  order  that  the  government 
shall  have  a  standard  against  which  to  measure  development 
environments  of  vendors. 

Before  a  model  is  admitted  to  general  use,  there  should  be 
adequate  testing  against  an  appropriate  set  of  different  software 
projects  in  order  to  adequately  describe  its  effective 
application  domain  and  the  expected  accuracy  of  the  model  in  that 
domain.  Desirable  characteristics  include: 

•  Automated  execution. 

•  Transportable  for  all  commonly  used  computers.  Written  in 
HOL . 

•  Model  algorithms  should  be  thoroughly  documented  and 
available  to  all  users. 

•  Outputs  should  be  flexible  and  tailorable  to  the  several 
applications . 

m  The  total  set  of  models  should  cover  the  whole  software 
life  cycle,  although  individual  models  may  be  specific  to 
certain  phases  of  the  life  cycle  or  certain  applications  of 
the  SCE . 

9  Models  should  be  able  to  deal  effectively  with  missing 
data  . 

•  Models  should  be  conservative  of  use  of  resources  for  data 
loading  and  computer  time. 

A  wide  variety  of  applications  of  SCE  models  exist,  and  a 
detailed  description  of  the  output  required  for  each  application 
is  impractical  in  this  paper.  Several  different  outputs  which 
should  be  considered  as  candidates  are  given  below.  Further 
study  should  be  undertaken  to  develop  a  more  specific 
description.  Each  of  the  items  listed  below  is  an  output  of  at 
least  one  SCE  model  which  exists  now  demonstrating  that  the  list 
is  not  unreasonable. 

•  Total  manpower  effort  by  phase  and  by  effort  type. 

•  Reasonable  development  time. 

•  Amount  of  documentation. 

•  Staffing  profile. 


•  Computer  costs  . 


•  Cost- schedule  trade-off  factors. 

•  Sensitivity  of  output  to  input  variations. 

•  Expected  rate  of  software  deficiency  repor  s. 

•  Milestone  occurence  times. 

•  Risk  profiles. 

SCE  models  must  be  designed  so  that  the  necessary  input  can 
be  reasonably  expected  to  be  available  when  the  model  is  applied. 
Most  existing  models  require  that  the  number  of  lines  of  code  be 
input.  Where  models  are  to  be  used  before  coding,  these  models 
are  not  very  helpful  because  a  difficult  manual  estimation  must 
be  accomplished  before  applying  the  model.  Models  which  use  more 
readily  available  information  are  urgently  needed.  For  example, 
models  which  use  the  size  and  complexity  of  a  language  for  the 
system  to  estimate  development  effort  would  be  very  useful .  It 
should  be  noted  that  some  applications,  e.g.  maintenance 
resource  requirements,  may  have  completed  code  metrics  available 
as  input  to  a  SCE  model.  Tools  are  needed  to  analyze  code  and 
determine  useful  metrics  such  as  number  of  lines  of  code, 
complexity  measures,  number  of  operators,  number  of  operands 
etc.,  which  can  be  used  to  estimate  maintenance  costs  or  to 
measure  software  quality. 

Conclusion  14:  Model  requirements  should  be  developed  in 
the  areas  of  input  and  output  parameters,  and  refined  to 
correlate  specific  requirements  with  each  anticipated  area 
of  model  application.  Research  should  include  establishing 
accuracy  requirements  in  each  instance,  and  determining  if 
the  required  accuracy  is  attainable. 

4 . 3  Software  Cost  Estimation  Metrics  and  Data  Collection 

In  just  about  every  piece  of  literature  or  special  report  on 
SCE,  a  common  conclusion  exists:  the  data  or  data  base  for  SCE 
is  in  bad  shape  in  that  the  data  are  non-homogenous .  Therefore, 
it  is  difficult  to  achieve  such  desirable  ends  as  model 
validation,  financial  performance,  technical  performance,  data 
base  development,  and  advanced  SCE  modeling.  In  addition,  the 
ability  to  try  other  predictive  metrics  is  missing,  thus  code 
size  is  usually  the  predominent  predictor.  Another  point  that  is 
clear  is  that  the  clarity  of  underlying  definitions  and  precision 
or  integrity  of  the  reported  data  is  poor.  Clearly,  there  isn't 
a  shortage  of  data.  However,  for  the  above  reasons  it  is 
generally  unuseable.  The  Panel  consensus  is  that  data  collection 
should  be  a  major  element  in  the  DoD  SCE  goals.  This  data 
collection  effort  should  be  well  defined,  properly  controlled, 
manageable  and  produce  the  highest  quality  data  possible. 

To  establish  data  collection  and  methodology  for  DoD,  an 
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implcmontat ion  approach  mist  be  established.  This  impl omenta tion 
approach  will  take  different  forms  as  the  SCE  technology 
improves.  Some  of  the  general  requirements  that  should  be 
present  in  any  implementation  approach  are: 

•  Data  collection  and  cost  estimating  methodology  should  bo 
well  defined  and  specified  in  appropriate  regulatory  form, 
i.e.,  in  DoDs ,  MIL-STDs,  DoD-STDs ,  DIDs ,  etc. 

•  A  centralized  group  should  be  provided  to  control  SCE 
methodology  application  and  development. 

•  Weapon  system  acquisition  Program  Management  Directions 
(PMDs)  should  specify  the  requirement  for  software  data 
collection.  In  addition,  the  PMD  should  allow  or  provide 
for  the  required  resource  to  do  that  data  collection. 

•  Flexible  accuracy  standards  should  be  developed  so  that 
the  results  of  SCE  activity  can  be  properly  evaluated  in 
view  of  the  current  software  LCC  phase.  For  each  phase  of 
the  software  life  cycle,  a  separate  accuracy  standard  should 
be  established  with  regard  to  the  results  of  the  software 
cost  estimate  for  that  phase.  These  accuracy  standards 
should  have  an  increasingly  finer  granularity  as  the  program 
moves  through  its  life  cycle. 

Software  metrics  have  been  produced  that  are  used  to  measure 
various  qualities  of  software.  The  quality  of  software  referred 
to  does  not  imply  "good"  or  "bad"  software.  Instead,  software 
qualities  are  those  attributes  of  software  by  which  we  determine 
software  reliability,  portability,  maintainability,  etc.  Work 
has  been  done  to  define  the  attributes  which  comprise  the 
respective  software  qualities.  A  f ew  automated  tools  have  been 
developed  to  make  the  data  collection  task  more  precise, 
manageable,  and  ] ess  time  consuming.  it  appears  feasible  to 
extend  the  software  quality  metrics  role  to  include  measures  that 
will  concentrate  on  attributes  which  are  the  "cost  drivers"  in 
system  life  cycle  phases.  In  certain  instances  the  attributes 
that  make  up  a  certain  quality  are  directly  applicable  to  the 
cost  elements  to  be  modeled.  If  a  system  is  complex  (new  and/or 
difficult)  then  the  cost  associated  with  the  system  over  its  life 
cycle  in  all  phases  should  be  higher  than  those  associated  with 
less  complex  implementations.  Algorithmic  complexity,  structural 
complexity,  and  performance  requirements  are  just  beginning  to  oe 
understood  and  measured.  Thus,  these  measures  offer  a  way  to 
quantify  the  cost  aspects  of  complexity  as  an  input  to  the 
modeling  situation.  Many  other  software  metrics  may  serve  as 
candidates  to  obtain  cost  parameters. 

Once  a  suitable  (beginning)  set  of  software  metrics  for  cost 
estimation  is  derived,  data  must  be  collected,  stored  in  a 
centralized  location,  and  analyses  conducted  that  will  result  in 
the  next  generation  of  more  precise,  accurate,  and  viable 


software  cost  estimating  methodologies.  This  gives  rise  to 
implications  for  both  management  and  technology.  A  uniform 
( standar i zed )  data  collection  instrument  must  be  designed  that 
v/ill  enable  data  collection  in  a  consistent  manner.  This 
approach  is  mandatory  to  avoid  problems  arising  over  which  data 
to  collect,  when  to  collect  them,  and  how  to  maintain  the  data  in 
machine  readable  format  for  subsequent  storage  and  analysis. 

A  candidate  Data  Item  Description  (DID)  has  been  prepared  by 
the  Air  Force  Systems  Command  Electronic  Systems  Division 
(AFSC/ESD)  which  specifies  magnetic  tape  formats  for  reporting 
financial  and  technical  data  on  system  acquisition  programs  under 
000  series  regulations.  The  DID  specifies  requirements  for  a 
Software  Cost  Performance  Report  (S/Vl  CPR)  to  be  used  in 
conjunction  with  a  companion  document  for  an  expanded  work 
breakdov.n  structure  (V.'DS).  The  S/M  CPR  is  an  automated  system 
which  produces  magnetic  tapes  in  accordance  with  the  DID  format 
instructions . 

Management  policy  is  needed  to  enforce  the  use  of  the  data 
collection  vehicle  and  sufficient  resources  allocated.  P 
repository  is  needed  to  focus  the  data  collection  activities. 
The  repository  could  also  be  chartered  to  perform  the  analytic 
functions  on  the  resulting  data  base.  Open  access  for 
technologists,  researchers,  management  personnel  should  be 
guaranteed  for  industry  and  government  alike.  Data  could  be 
"sanitized"  to  remove  the  onus  of  competitive  advantage  and 
possible  revelation  of  corporate  cost  estimating  strategies.  h 
very  positive  benefit  is  obtained  by  allotting  a  viev/  across  the 
data  base  and  more  quickly  obtaining  a  critical  mass  of  data  for 
cost  modeling  and  methodology  development.  One  possible  choice 
for  such  a  repository  is  the  Data  and  Analysis  Center  for 
Softv/are  (DACS)  located  at  the  Rome  Air  Development  Center.  DACE, 
is  presently  responsible  for  collecting  and  disseminating 
software  data  and  could  be  called  upon  to  perform  an  expanded 
role  for  cost  estimation.  In  Government  fiscal  year  1903,  the 
DACS  v/ill  be  operated  under  the  aegis  of  the  Defense  Technology 
Information  Center  system  of  Information  Analysis  Centers  (IACs) 
for  the  Defense  Logistics  Agency. 

The  ultimate  unit  of  software  selected  for  measurement  and 
data  collection  will  vary  depending  on  the  life  cycle  phase  of 
interest  and  the  particular  cost  estimating  methodology  and 
models  employed.  Thus,  the  unit  of  software  at  any  point  in  time 
should  be  considered  as  a  variable.  During  the  conceptual  phase 
of  the  life  cycle  a  less  detailed  view  of  software  is  needed  than 
during  the  full  scale  development  phase  where  functional 
allocation  has  been  made  to  computer  program  configuration  items 
(CPCI's)  and  lower-  Models  used  during  early  phases  from  concept 
through  preliminary  design  v/ill  utilize  software  units  where 
scale  factors  are  more  important  than  implementation  details. 
Later  in  the  life  cycle,  cost  models  may  be  employed  for  cost 
performance  and  tracking.  The  granularity  of  software  units  and 
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cost  elements  will  correspond.  Technology  for  metrics  and  data 
collection  will  need  to  account  for  the  varying  qranul nr ity  of 
software  components . 

A  work  breakdown  structure  (WBS)  to  support  metric  and  data 
collection  is  needed.  The  current  version  of  MIL-STD  C^iA 
specifies  a  UBS  for  system  acquisition  with  elements  at  the 
first,  second,  and  third  levels.  Additional  levels  of  detail  in 
the  W3S  are  needed  if  effective  cost  model inq  is  to  be  obtained. 
Software  related  UBS  elements  must  account  for  operational 
computer  software,  computer  programs  that  support  the  acquisition 
of  operational  software,  and  software  that  support  the  services, 
facilities,  and  data  required  to  acquire  the  defense  system.  A 
draft  military  specification  has  been  prepared  by  BSD  which 
expands  the  current  UBS  to  the  fourth,  fifth,  and  sixth  levels. 
This  UBS  will  be  applied  by  ESD  on  n  pilot  basis  in  the  near 
future  and  should  be  monitored  for  suitability  and  performance 
vis-a-vis  cost  estimation.  Consideration  should  also  be  given  to 
a  companion  document  prepared  for  F.SD  which  establishes  reporting 
requirements  needed  to  characterize  computer  software  developed 
during  system  acquisition.  This  document  should  require  that 
software  data  be  collected  on  structure  (i.e.  functionality), 
schedules,  factors  indicating  size,  degree  of  difficulty,  numbers 
of  development  personnel,  development  changes  from  initial  plans, 
and  deviations  from  accepted  practices.  These  data  could  then  be 
used  to  monitor  software  acquisition  in  conjunction  with  the 
reported  software  cost  data.  The  cost  data  will  explicitly 
identify  cost  problems,  and  the  analysis  of  cho  other  data  should 
provide  reasons  for  cost  problems.  In  view  of  the  poor 
performance  of  existing  software  cost  estimation  models  and 
methodology,  it  is  clear  that  a  better  understanding  of  the 
software  life  cycle  is  needed  in  order  to  establish  new  insights 
into  the  process  for  cost  estimation  purposes.  Technology 
improvements  are  needed  for  cost  modeling,  but  these  improvements 
will  not  be  realized  unless  there  is  a  corresponding  improvement 
in  software  metrics  and  data  collection.  These  tasks  are  the 
foundation  by  which  more  precise,  accurate,  and  timely  cost 
estimation  can  be  performed. 

The  above  discussion  supports  the  following  conclusions: 

Conclusion  15:  Additional  research  in  the  area  of  software 
metrics  is  needed  to  define  software  attributes  v.'hich  are 
the  cost  drivers  over  the  life  cycle. 

Conclusion  16:  Data  collection  activities  must  be 
established  using  the  metrics  in  an  organized  and 
standarized  manner . 

Conclusion  17:  Data  collection  should  be  automated  and 
outputs  provided  in  machine  readable  format. 

Conclusion  IB:  A  central  repository  should  be  designated 


for  storing  and  analyzing  software  cost  estimation  data. 


Conclusion  ln:  f'lIL-CTD-^Tl A  should  be  modified  to  permit  a 
work  breakdown  structure  that  supports  data  collection  and 
analysis . 

4.4  Contractual  Implications  of  Applying  (SCEl  to  Contractor 
Per formance 

4.4.1  General  Contractual  Considerations 

• 

The  Government  uses  various  tools  to  ascertain  the 
contractor's  performance  in  ful fulling  the  terms  of  a  contract. 
Information  derived  from  these  tools  might  or  might  not  be 
combined  with  SCE  methodology  to  derive  an  additional  view  of 
contract  performance.  Three  areas  have  been  identified  where  the 
SCE  methodology  may  play  a  significant,  contractual  role.  These 
areas  are:  technical  performance  measurement  (TPM),  progress 
payments,  and  fee  determination. 

In  general,  potential  contractual  problems  could  exist  if 
specific  SCE  methods  were  imposed  by  the  Government.  If  the 
Government  imposed  a  type  or  types  of  models  for  use  by 
contractors,  there  would  be  immediate  problems  with  validating  or 
calibrating  the  methodology  for  the  contractor's  environment. 
This  would  lead  to  misinterpretations,  inaccuracies,  and 
potential  legal  problems  when ,  or  if,  the  SCE  methodology  would 
be  applied  to  a  performance  measurement  situation.  Hence,  it 
would  be  in  the  mutual  interest  of  the  Government  and  industry  to 
constrain  contractual  application  of  SCE  methodologies  to  a  set 
of  narrow  guidelines. 

If  the  Government  desires  the  use  of  SCE  methodology,  then 
the  contract  should  only  specify  that  the  contractor  will  define 
the  particular  SCE  methodology,  that  the  particular  SCE 
methodology  will  be  analytical  in  structure,  and  that  both 
parties  will  mutually  agree  to  the  minimum  input  and  output 
parameters.  Provisions  should  also  be  made  to  assure  that  the 
contractor's  SCE  methodology  could  sustain  modest  refinement 
(i.e,  change  to  factors  or  structure),  but  prohibit  major  or 
radical  method  changes  without  a  two  party  agreement.  The  use  of 
the  SCE  methodology  by  the  contractor  should  be  oriented  more  as 
a  tool  for  the  contractor  to  communicate  to  (is  managers  and 
developers  in  addition  to  the  Government.  Thus,  the  SCE 
methodology  would  best  function  if  integrated  into  the 
contractor's  management  system. 

The  first  area  where  SCE  methodologies  could  be  applied  to 
the  performance  measurement  process  is  the  technical  performance 
measurement  program,  as  defined  in  MIL-STD-499A .  Software  cost 
estimation  methodologies  would  be  a  reasonable  adjunct  to  the 
evaluation  of  technical  parameters,  particularly  v/hen  risk 
assessment  (costs  and  schedule)  is  needed.  The  addition  of  a  SCE 
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methodology  would  provide  a  cons  i  stent ,  timely  nml  cost-effective 
means  of  including  management  factors  with  technical  performance 
factors.  It  would  also  assure  a  quantitative  feedback  so  that 
reasonable  priorities  and  tasks  could  be  formed  to  address 
identified  problem  areas.  The  SCF,  methodology  would  aid  in 
communicating  to  all  parties  the  scope  of  a  problem. 

Conclusion  20:  The  Government  needs  to  integrate  the  SCF. 
methodoloqy  requirement  into  its  technical  performance 
measurement  system.  Since  the  application  of  the  SCE 
methodology  is  strictly  informational,  no  contractual 
problems  would  result.  The  contractor,  however,  must  be 
given  the  latitude  to  select,  nodify,  or  develop  the 
evaluation  methodology  that  is  best  suited  for  the 
particular  procurement  environment. 

Progress  payments  represent  another  area  where  SCE  methodologies 
could  have  a  significant  contractual  impact.  A  software  model 
(or  models)  could  be  used  to  compare  predictions  with  actual 
cost/schedule  performance  independently  of  any  cost  tracking 
system  (e.g.,  C/SCSC  or  C/SSR).  The  results  of  such  a  comparison 
could  give  management  the  basis  to  reevaluate  contract 
performance.  However  it  appears  inappropriate  for  the  Government 
to  use  such  results  in  a  progress  payment  determination.  SCE 
methods  presently  have  not  demonstrated  sufficient  accuracy  to 
serve  as  criteria  for  payment  or  nonpayment.  Also,  cost 
estimates  tend  to  be  real-time  and  do  not  reflect  established 
baselines.  It  is  more  appropriate,  then,  to  expect  the  software 
cost  estimation  techniques  to  be  predictive  of  future  performance 
and  not  applicable  to  evaluating  past  performance. 

Conclusion  21;  The  GCE  methodologies  should  not  be  applied 
to  ,  ogress  payment  determinations.  Such  methodologies  are 
relatively  inaccurate  and  only  predictive  in  nature. 

In  establishing  a  fee  payment  structure,  SCE  methodologies 
could  possibly  be  applied.  For  instance,  the  requirement  for 
stating  and  meeting  cost  and  schedule  objectives  could  be  tied  to 
a  fee  payment.  This  would  give  an  incentive  for  the  contractor 
to  avoid  misleading  or  misrepresenting  management  and  technical 
factors  that  make  up  a  forecast.  However,  as  was  the  case  for 
progress  payments,  one  recognizes  the  inaccuracies  associated 
with  present  d ■  'CE  methodologies  are  only  predictive  in  nature. 
Hence,  '1  wou]  '  ..cm  inappropriate  for  such  methods  to  be  applied 
or  connected  with  fee  determinations.  The  present,  appropriate 
approach  is  to  use  metric  methodologies  which  rely  on  measures  of 
past  events  . 

Conclusion  1  '  SCE  methodology  should  not  be  employed  in 
the  fee  c  .rmination  process  ns  it  is  predictive,  and 
cannot  adequately  support  evaluations  of  predetermined 
baselines  and  criteria. 
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4.^.?  Program  r'anngcmont  Considerations 

An  accurate  picture  of  projected  costs  and  schedules  is 
essential  for  effective  program  management.  Every  nrogram 
manager  would  prefer  to  have  projections  that  are  based  on  the 
best  and  most  current  data  available.  Such  information  also  can 
serve  as  a  valuable  contractor  tool  for  assessing  the  need  for 
reallocation  of  resources.  The  desire  for  the  best  available 
information  must  be  balanced  against  the  cost  to  obtain  such 
information.  It  is  apparent  that  the  need  for  updated  cost 
estimates  will  vary  with  the  size  of  the  software  development 
effort  and  the  level  of  risk  associated  with  the  development.  It 
is  unreasonable  to  require  a  small,  straight  forward  project  to 
bear  the  cost  burden  of  extensive  data  collection  and  SCE 
computation.  However,  every  program  involving  software 
development  should  gather  software  cost  estimating  data.  Data 
should  be  collected  at  the  end  of  each  phase  of  the  development 
life  cycle. 

A  definite  requirement  exists  for  regularly  scheduled 
updates  to  cost  and  schedule  estimates.  The  development 
contractor  should  be  obligated  to  provide  the  necessary  data,  and 
funding  should  be  provided  for  this  purpose.  Since  critical 
program  decisions  often  are  associated  with  results  from  formal 
program  reviews,  these  program  review  points  constitute  logical 
points  for  Government  review  and  recalculation  of  SCEs . 

For  programs  that  can  be  identified  as  involving  a  high 
degree  of  risk,  due  either  to  technical  complexity  or  uncertainty 
in  original  costs,  contractor  recalculation  of  cost/schedule 
estimates  should  be  required  in  conjunction  with  each  formal 
program  review.  This  additional  point  of  comparison  should 
assist  the  Government  in  the  monitoring  of  high  risk  efforts. 

In  addition  to  the  scheduled  recalculation  of  FCEs  in 
conjunction  with  formal  program  reviews,  recalculations  should  be 
required  whenever  major  changes  or  deviations  occur  in  a  program. 
Examples  of  such  changes  include  modifications  to  the  program 
scope,  budget  reductions  or  changes  in  the  development 
environment.  ETien  such  changes  are  proposed  by  the  contractor, 
change  proposals  should  be  accompanied  by  cost/schedule 
recalculation  using  the  contractor's  RCE  methodology.  In  all 
cases,  the  contractor  should  be  obligated  to  provide  necessary 
input  data  and  the  Government  should  recalculate  costs/ schedules . 

The  balance  betv'een  the  need  for  current  estimates  and  the 
cost  to  obtain  such  estimates  will  change  as  collection  methods 
become  more  efficient  and,  thus,  less  costly.  The  best 
opportunity  to  reduce  collection  costs  involves  increased  use  of 
automation  in  the  collection  effort. 

Conclusion  23:  ECE  data  should  be  collected  at  each  formal 

program  review  point.  These  data  can  be  gathered  by 
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automata  or  other  "no  cost"  neans .  Successful  application 
depends  on  cost  efficient  (automated)  collection  methods. 


4.5  Software  Cost  Estimation  Technoloqy  Development  Plan 


The  first  stop  in  establishing  an  effective  DoD  technology 
plan  is  to  clearly  state  DoD  goals.  A  DoD  standard  and  guidebook 
is  in  order  to  insure  a  standard  methodology  for  collecting  and 
categorizing  data  for  the  system  and  software  data  base.  The  DoD 
standard  would  also  insure  commons] ity  for  the  data  base  item 
descriptions  and  terminology.  The  DoD  standdard  would  be  used  by 
both  Government  and  industry  to  insure  proper  identification  of 
data  for  the  data  base  by  system  type,  development  methodology, 
complexity  ,  schedule  duration,  etc.  It  is  also  anticipated  that 
a  guidebook  will  be  required  to  describe  ho w  to  access  and  use 
the  data  base  and  how  to  utilize  the . cost  models  developed  from 
the  data  base.  The  guidebook  should  contain  procedures  for  using 
the  various  models  of  software  costs  and  should  indicate  how 
these  models  may  be  tailored  for  a  particular  application. 


The  embracing  of  FCE  methodologies  will  be  an  evolutionary 
process.  Policy  should  be  in  conformance  with  and  serving  to 
implement  an  overall  SCE  technology  plan.  This  policy  should  be 
initiated  by  JLC  with  a  policy  statement  that  addresses  the 
following: 


•  Intent  and  goals  with  respect  to  SCE  technology 

•  RFP  consideration 


•  Gource  selection 

•  Data  acquisition 

•  Contractor  performance  evaluation 

•  Use  by  life  cycle  phase 

•  Model  technology  development 

•  Regulatory  policy  requirements  from  DoD 
,  •  Protection  of  proprietary  information 

•  Prime/subcontractor  interrelationship 

•  Scope  of  programs  affected 


As  the  use  of  SCE  methodology  matures,  additional  guidance 
can  be  prepared  and  disseminated.  This  could  be  in  the  form  of  a 
DoD  directive,  guide  books,  central  data  repositories,  etc. 


The  Panel’s  assessment  of  current  SCE  technology  reveals  an 


-34- 


absolute  need  for  improved  methodology  to  accurately  predict  the 
software  resources  required  over  the  complete  software  life 
cycle.  As  advances  in  software  engineering  are  made,  those 
advantages  should  be  reflected  and  incorporated  in  the  improved 
SCE  methodologies.  The  SCE  technology  also  should  be  useful  to 
all  of  the  large  numbers  of  potential  users,  each  of  whom  has  a 
diversity  of  application  for  this  technology.  An  undertaking  to 
collect  statistically  valid  historical  data  is  the  next  logical 
step  in  developing  better  SCE  models.  This  effort  should  begin 
as  soon  as  possible  and  should  be  accomplished  as  a  requirement 
of  all  software  related  contracts  and  activities.  It  is 
desirable  that  a  central  DoD  agency  be  identified  to  provide  the 
proper  incentive,  direction,  and  control  for  this  effort. 

A  number  of  activities  are  proposed  as  a  plan  to  achieve 
both  an  interim  capability  and  the  long  term  development  of  a 
complete  SCE  methodology.  A  phased  development  plan  is 
considered  appropriate  to  achieve  near-term  and  long  term  aoals. 

In  the  near  term  (next  three  years),  DoD  should  concentrate 
on  implementing  procedures  to  best  use  FCE  methodology  as  it  now 
exists,  and  establish  a  data  base  suitable  to  develop  advanced 
SCE  methodology.  Factors  to  be  reviewed  includes 

•  A  guide  to  applying  existing  SCE  methodology. 

•  A  DoD  policy  for  use  of  SCE  methodology. 

•  Establishment  of  a  DoD  control  center  for  technology  and 
ssociated  working  groups  to  encourage  research  in  SCE 
technology  in  the  form  of  alternate  metrics  and  advanced 
simulation  models. 

•  Definition  of  the  metrics  and  data  required  for  the 
collection  procedure  that  has  provisions  for  changing 
collection  requirements . 

•  Implementation  of  data  collection  procedures  that  have 
provisions  for  changing  collection  requirements. 

•  Reevaluation  of  the  goals  and  objectives  for  SCE 
technology . 

On  a  broader  front,  DoD  should  concentrate  on  gaining  an 
accepted,  standard  SCE  methodology  for  general  application.  This 
could  be  achieved  in  three  to  seven  years.  Goals  at  this  level 
should  include: 

•  Development  of  improved  methodology  for  software  resource 
estimates  (e.g.  develop  SCE  model  inputs  for  program  size 
estimation  based  program  requirements,  languages,  or  program 
design  languages) . 
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•  Validation  of  existing  models  based  on  the  data  base 
acquired  in  the  near  term. 

•  Establishment  of  prototypes  of  advanced  SCE  methodology. 

•  Implementation  of  improved  SCE  methodology  to  gap  the 
existing  versus  the  complete  methodology. 

•  Establishment  of  guidel ines/procedures  for  the  application 
of  the  complete  SCE  methodology. 

•  Development  of  complete  SCE  methodology. 

Conclusion  24t  The  JLC  should  issue  a  policy  that 
implements  an  SCE  direction.  This  policy  should  direct  that 
SCE  methodology  be  adopted  that  makes  existing  SCE 
technology  usable  by  program  managers.  It  should  also 
provide  for  a  technology  upgrade  to  SCE  methodology  over  the 
next  seven  years. 

5.  RECOMMENDATIONS 

The  Panel  has  developed  four  recommendations  for  the  Joint 
Logistics  Commanders  as  a  means  to  best  utilize  existing  software 
cost  estimating  technology  in  improving  software  cost  estimates, 
and  a  program  to  follow  to  gain  better  and  more  accurate  cost 
estimates  in  the  future. 

Recommendation  Is  Use  of  Existing  SCE  Models 


The  Panel  believes  that  no  existing,  SCE  model  is  sufficient 
to  adapt  as  an  embedded  computer  system  standard.  The  Panel 
recommends  that  the  JLC  not  adapt  any  existing  SCE  model  as  a 
standard.  (Refer  to  conclusions  1,  2,  3,  4,  5,  f,  0,  and  u )  . 

Recommendation  2;  Application  of  Exist. inn  SCE  Methodology 

The  Panel  believes  that  a  judicious  use  of  SCE  models  and 
methodology  can  improve  the  acquisition  and  management  of 
softv/are.  There  is  no  accepted  methodology  or  guide  to  using 
current  SCE  models  or  methodology  in  a  generally  acceptable 
manner.  The  Panel  recommends  that  a  guidebook  be  developed  that 
can  be  used  by  program  offices  to  orderly  qualify  models  and 
methodologies  to  develop  better  softv/are  cost  estimates 
throughout  the  entire  software  life  cycle.  (Refer  to  conclusions 
7,  13,  19,  20,  21,  22,  23,  and  24). 

Recommendation  3 ;  Improving  SCE  Methodology 

The  Panel  believes  that  an  SCE  methodology  can  be  developed 
that  would  result  in  SCE  models  that  could  be  generally  applied 
by  both  the  Government  and  industry  to  estimate  embedded  computer 
system  software  costs  well.  The  Panel  recommends  that  JLC 
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sponsor  a  progrnn  to  implement  an  improved  OCR  methodology. 
(Refur  to  conclusions  10,  11,  12,  13,  14,  15,  and  24). 

Recommendation  4:  P.stabl  ishinq  an  SCE  Data  Rase 

The  Panel  believes  that  an  improved  software  cost  estimating 
methodology  must  be  supported  by  the  gathering  and  maintenance  of 
an  accurate,  complete,  and  coherent  data  base.  The  Panel 
recommends  that  JLC  appoint  an  existing  Government  agency  ns  an 
SCE  data  base  repository  and  empower  this  agency  to  develop  data 
collection  standards.  (Refer  to  conclusions  15,  16,  17,  10,  19, 
and  23 ) . 
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INTERCOM  SYSTEMS  CORPORATION 


14  May  1981 

To:  Members  of  the  JLC  Panel  on  Estimating  Software  Costs 


Thank  you  for  agreeing  to  serve  on  the  Estimating  Software  Costs 
Panel  of  the  Joint  Logistics  Commander  Joint  Policy  Coordinating 
Group  on  Computer  Resource  Management  (JPCG-CRM) .  Bob  Berri  and 
I  look  forward  to  participating  with  you.  As  you  all  know,  the 
workshop  will  take  place  in  Monterey,  from  22  June  through  25 
June.  The  tentative  agenda  shows  registration  on  22  June  between ' 
1300  and  1500,  followed  by  a  general  work-shop  session  until  1715. 
At  that  time,  we  will  meet  as  a  panel  for  a  get  acquainted  and  a 
get  organized  session.  Thereafter,  we  will  mostly  meet  as  a  panel 
until  presenting  our  results  at  a  general  work-shop  session  at  1500, 
25  June. 


Attached  is  a  copy  of  suggestions  for  the  panel  that  was  given  to 
us  by  JPCG-CRM,  which  effectively  serves  as  a  charter  for  our  panel. 

Our  expected  results  might  very  well  be  broken  down  to  another  | 

level  of  detail,  as  follows: 

1.  Evaluation  of  pertinence  of  existing  models. 

2.  Evaluation  of  limitation  of  existing  models. 

3.  Determination  of  desired  estimation  model  output 
by  software  life  cycle  phase. 

4.  Determination  of  minimum  parameters  to  serve  as 
input  for  estimation  models. 

5.  Determination  of  acceptable  interfaces  for  exchang- 

ing  information  between  DOD/industry  for  bidding,  " 

contracting,  and  monitoring  purposes. 

6 .  Recommendation  of  R  &  D  direction  to  improve  i 

technology. 

Most  of  you  will  be  somewhat  familiar  with  the  various  estimation 

models  now  published.  A  bibliography  is  attached  for  those  of  you 

who  wish  to  do  some  homework  before  the  work-shop.  In  addition  to  I 
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this,  we  solicit  all  of  you  to  bring  any  of  your  own  estimating 
procedures,  present  or  future  needs,  results  (good  and  bad)  from 
past  efforts,  thoughts  for  borrowing  from  other  technologies,  et 
cetera  to  share  with  the  panel. 

Our  panel  will  have  approximately  18  people  on  it.  (Attached  is 
a  list  of  the  members  and  their  affiliations  for  your  interest) . 

As  a  result,  we  will  likely  want  to  split  our  effort  into  three 
or  four  narrower  issues,  with  subpanels  addressing  themselves 
to  these  issues.  Each  of  you  might  give  this  same  thought,  and 
we'll  try  to  decide  on  the  subpanels  at  our  get  organized  meet¬ 
ing.  Some  suggestions  for  subpanel  topics  that  have  been  ad¬ 
vanced  are: 

•  Evaluation  of  existing  model  strengths  and  limita¬ 
tions. 

§ 

•  Recommendations  for  R  &  D  technology  directions. 

•  Definition  of  use  of  estimation  model  in  software 
life  cycle  activities. 

•  Implications  of  estimation  model  use  in  contractual 
relations. 

Our  panel  has  been  assigned  a  topic  that  has  tradionally  received 
a  lot  of  attention,  occupied  many  pages  of  literature,  and  has 
generally  fallen  into  the  category  of  black  magic.  It  will  be  a 
real  challenge  to  see  if  our  three  days  of  effort  can  advance  the 
state  of  art.  Bob  and  I  believe  that  we  can,  given  the  collective 
talent  and  experience  that  is  being  brought  together.  We  look 
forward  to  interesting  sessions  with  you  all  in  Monterey.  If 
you  have  any  questions,  please  call  either  Bob  or  me,  or  the  work- 
ship  coordinator,  2Lt.  Ed  Petersime,  Hq.  AFLC/LOEC  at  (513)  257-2054 
or  AV  787-2054. 

Very  truly  yours. 


R.  Dean  Hartwick 

RDH: lb 

cc:  Ms. 

Mr. 

Mr. 

Mr. 

Mr. 
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JUitonia  Schuman 
Dick  Maher 
Robert  Dunn 
Robert  Berri 
Jack  Munson 


Maj.  Larry  Fry 
CDR  Ronald  Oh lander 
LCDR  John  Barnes 
Dr.  Matt  Fisher 
LTC  Casper  Klucas 


SUGGESTIONS  FOR  A  MONTEREY  PANEL  ON 
ESTIMATING  SOFTWARE  COSTS 


1.  Issue:  Given  the  increasing  percentage  of  total  system  cost 
that  must  be  devoted  to  software#  it  is  becoming  more  critical 
that  viable  procedures  be  developed  for  accurately  predicting 
the  cost  of  software.  Is  this  possible  using  the  Software  Life 
Cycle  Cost  Models  currently  available? 

I  2.  Questions; 

[  —  How  and  where  are  current  Software  Life  Cycle  Cost  Models 

|  of  benefit? 

(j  i. 

|  —  For  which  phases  of  system  development  can  Software  costs 

I  be  accurately  estimated?  (V7hy?) 


For  which  phases  of  system  development  can  Software  costs 
not  be*  accurately  estimated?  (Why?) 

Is  the  use  of  program  size  or  lines  of  code  an  accurate 
guage  for  estimating  costs? 


Expected  Results: 


—  An  evaluation  of  Software  Life  Cycle  Cost  Models  i;denti^ 
fying  deficiencies  or  strengths  of  each. 

u  , 

—  Recommendation  on  how  to  improve  Software  cost  estimating. 
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A  MACHINE  INSTRUCTION 


DEFINITION 


A  .‘.ACHlui  US  I  RUCTION  IS  A  CODE  THAT  CAUSES 
PROCESSOR  TO  PERFORM  A  SINGLE  OPERATION. 
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bUGLL  PROCESSOR,  Slo.  PERIPHERALS 
3 1  NO LE  FROCESSOR,  CXTENULD  PERIPHERALS 
MULT  I -PROCESSOR,  STii.  PERIPHERALS 
KULT1 -  PROCESSOR,  COMPLEX  PERIPHERALS 
DISTRIBUTED  HUT  LI  PROCESSOR  SYSTEMS 
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SPACE,  UAL  TIME 
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rilrtu.(  RUE  3  ION  OF  a  FAMILIAR  SYS* 
“CURATE  KC!)i.3!u*i  Or  A  FAMILIAR  SYS. 
I'.AJuR  REDESIGN  Gk  HEW  FAMILIAR  SYS. 
I.uLFLA TcLY  !I-»VLL  DESIGN 
REVOLUTIONARY 

iu,  ,  . L  i  Sl  L  Ml  IliUOuLOOY  UULS  i  1UI.UA1KL) 
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jlLIuL  r'.ATlIKlIY 

ii'J  CiiriiHiL  S  1 

FEW  UihNOLS  2 

MCJEKhl l  CHANGES  3 

US  1‘AbLL/riAiJY  CHANGES  4 

CUiJCORi  LuT  PROCuSSOfv  jLSIGN  5 
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A  MEIHUUULUuY  HUE  ST  1UN1JA IRE 

I  JS  Tilt  SuFTWAKE  OFF  Tilt  CRITICAL  SCHEDULE  PATH? 

I  IS  hiii  uPlNaTIhU  LYSItii  AVa  I  LADLE? 

I  lb  TliL  IIA Kbit/vKc.  PKUCLSSUiv  AVAI  LADLE? 

I  Is  I ;ic  bLbiub  ALLUCATlB  TO  MODULES? 

•  lb  TOP-DOWN  DESIGN  PLANNED? 

•  IS  TOP-DOWii  hiTCGluaiOH  PLANNED? 

I  ARE  PROJECT  IJUT tbUuKS  PLANNED? 

•  i\i,l  COLE  INSPECTIONS  PLANNED? 

5  AKl  Sui'h.V.RE  TEST  PLANS  FO EMULATED? 

i  /n.L  >\j i uric, i i.u  ksT  i'jcib  Planned? 
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I  lP  I).  CALCSlmYc  PRuDuClTiYiTY  FROM  Tilt  S/STEM  COIIPLEX1IY 
iliuEX  rKorlLt .  COMPARE  WITH  AH  ASSESSMENT  Ur  SYSTEM 
UlFFituLTY  AS  "VEkY  IIAiiD"  (PI  »  0.^  liJSTR. 7M-HR. ) , 
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Li  x  ,,x,  WHERE  R1  D  AN  APPROPRIATE  RUNNING  RATE  FOR 
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BUUGtTAKY  ESTIMATE  PROCEDURE  (CUNT . ) 


STEP  11.  ESTIMATE  CuMPUTER  SUPPORT  COSTS  AS  SI  =  COMPUTER 
FACTOR  x  LI  x  HI  (Si  IN  $). 

STEP  12.  ESTIMATE  DOCUMENTATION  COST  AS:  Nl/S  x  Dl,  WHERE  D1  IS 
$/PAGES. 

STEP  15.  ESTIMATE  TRAVEL  T1  IN  $. 

STEP  14.  COMPUTE  SOFTWARE  COST  AS:  Cl  -  LI  x  R1  +  SI  +  Dl  x 
N 1/5  +  Tl. 
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•  COMPLEXITY 
I  SIZE 

•  MODULARITY 

•  DIFFICULTY 
I  NUVELTY 

•  METHODOLOGY 

I  PKUCESSOR  DESIGN  MATURITY 

•  CPU  RESOURCE  UTILIZATION 

•  ADDED  COSTS 

•  RISKS  -  POSSIBLE  ADDED  COST  FACTORS 
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ADDED  COSTS  ESTIMATE 


STEP  1.  IDENTIFY  ALL  ADDED  COST  ITEMS,  e-g-: 

I  SUB-CONTRACTED  ITEMS 

•  LEASED  COMPUTER  SUPPORT  SYSTEMS 

•  SPECIAL  FACILITY  REQUIREMENTS 

•  LEASED  COMMUNICATIONS 

I  CAPITAL  SOFTWARE  DEVELOPMENT  SUPPORT  SYS. 

•  OTHER  COSTS  NOT  COVERED  ELSEWHERE 

STEP  2.  ESTIMATE  EACH  ADDED  COST  ITEM  IN  $. 

STEP  3.  SUMMARIZE  ADDED  COSTS,  CA  IN  $. 

STEP  A-  UPDATE  SOFTWARE  COST  ESTIMATE,  C3  -  C2  +  CA 
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.  COST  INFLUENCES 

I  COMPLEXITY 

•  SIZE 

•  MODULARITY 
I  DIFFICULTY 

•  NOVELTY 

•  METHODOLOGY 

•  PROCESSOR  DESIGN  MATURITY 

I  CPU  RESOURCE  UTILIZATION 
•  ADDED  COSTS 

I  RISKS  -  POSSIBLE  ADDED  COST  FACTORS 


RISKS  -  P0SS1BLL  AUDLD  GUST  FACTORS 


OMITTED  OR  INCORRECT  REQUIREMENTS 
SYSTEM  RELIABILITY  DEMO  REQUIRED 
CRITICAL  PROGRAMS  TO  BE  CFE 
CUSTOMER  IMPOSED  QA  PROVISIONS 
CRITICAL  DATA  BASES  TO  BE  CFE 
TEST  DATA  TO  BE  CFE 

CRITICAL  COMPONENTS  TO  BE  DEVELOPED  BY  YOU  AT  CUSTOMER 
FACILITY 

KEY  PERSONNEL  TO  BE  CFE 
SPATIALLY  SEPARATED  DEVELOPMENT  FACILITY 
UNUSUALLY  INEXPERIENCED  PERSONNEL 
APPLICATION  IS  HIGHLY  CLASSIFIED 
HARDWARE  PROCESSOR  CHANGE  IN  MID-PROJLCl 
FIXED  PRILL,  InLLNNVL  OR  PENAL!  Y  CUNIKACI 
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REASONABLENESS  CHECKS 


I  A  BUDGETARY  ESTIMATE  MAY  BE  USED  AS  A  REASONABLENESS 
CHECK  AGAINST  ANOTHER  ESTIMATION  METHOD. 


I  A  BUDGETARY  ESTIMATE  MAY  BE  COMPARED  FOR  REASONABLENESS 
AGAINST  PREVIOUS  EXPERIENCE  WITH  SOFTWARE  DEVELOPMENT 
PROGRAMS  FOR  SIM  I  UR  SYSTEMS- 
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A  SOFTWARE  DEVELOPMENT 


EFFORT  MODEL* 


DEVELOPMENT  PHASE 

IND#* 
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TRW** 

(SCIENTIFIC! 

DADS 

(FORECAST) 

PROPOSED 

MODEL 

ANALYSIS  AND  DESIGN 

52X 
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50 

AND  DOCUMENTATION 

CODING 

1GX 

2M 

21X 

20 

INTEGRATION  AND 

32X 

23  X 

31X 

30X 

TEST  I  NO 


•  LXCLUU1NG  PROPOSAL  AND  FIELD  SOPPURI. 

**  IRW  SYSTEMS  DIVISION,  "THE  COST  OF  DEVELOPING  LARGE“SCAI E 
SOFTWARE/  RAY  WULVERTON,  ILEET  VOL.  C-23,  No.  G,  JUNE  1974- 
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(10  DLL  IMPLEMENT  AT  ION 


THE  MODEL  IS  CURRENTLY  IMPLEMENTED  ON 

•  Tilt  TI-5D  HAND-HELD  PROGRAMMABLE  CALCULATOR 

-  USES  ALL  OF  MEMORY 

-  USER  INTERFACE  NOT  PARTICULARLY  FRIENDLY 
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•  THE  TI  990/10  AND  990/12  MINICOMPUTER 
-  BETTER  HUMAN  INTERFACE 
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■  Li  -tir*  riF 


1.0 


OBJECTIVE 


The  Software  Reusability  Panel  was  convened  to  evaluate  whether  reusabil¬ 
ity  represents  a  potentially  valuable  concept  to  reduce  cost  and  elapsed 
time  to  develop  embedded  computer  system  software.  Further,  if  there  was 
£  consensus  on  the  viability  of  this  issue,  then  what  barriers  must  be 
overcome  and  how  does  the  program  manager  and/or  software  manager  make 
reusability  a  reality? 


2.0 


SCOPE 


A  growing  concern  exists  in  DOD  and  in  industry  over  the  lack  of  reappli¬ 
cation  of  software  and  the  difficulty  in  adapting  software  to  even  very 
similar  systems.  By  contrast,  standardization  in  the  hardware  arena  has 
already  yielded  DOD  gains  in  various  aspects  of  life  cycle  costs.  In  the 
commercial  world ,  reuse  of  software  has  been  widespread  and  profitable 
for  many  companies.  These  lessons  need  to  be  drawn  upon  by  the  DOD 
software  community  to  achieve  parallel  gains  in  this  critical  area. 

Two  other  dimensions  are  discernible  as  stimuli  toward  achievement  of 
reusability.  First,  software  applications  are  growing  at  a  tremendous 
rate  in  both  size  and  complexity.  This  growth  rate  will  far  outstrip  the 
capacity  of  our  universities  to  turn  out  software  engineers.  A  way  must 
be  found  to  produce  more  software  with  a  small  increase  in  staff.  An 
obvious  solution  would  be  to  find  ways  to  reapply  existing  software 
rather  than  to  create  new  software  for  each  new  application.  Secondly, 
the  technology  trends  towards  microcomputers  and  very  large  scale 
integration  are  going  to  push  for  complex  embedded  software  implementing 
specific  functions  in  these  small  hardware  elements.  When  this  occurs, 
software  must  be  reusable  in  order  for  the  hardware  to  be  extendable  into 
many  different  applications.  Conversely,  if  the  software  does  not  become 
reusable,  the  gains  in  life  cycle  cost  and  risk  reduction  in  DOD  embedded 
systems  acheived  via  hardware  standardization  will  be  eroded. 

Three  specific  types  of  reusability  were  surfaced  and  addressed  by  the 
panel : 


1)  Reusing  functional  software  systems  across  multiple  configurations  of 
the  same  basic  system.  An  example  of  this  is  the  variation  which 
occurs  between  different  platforms  supported  by  the  Naval  Tactical 
Data  System  Software. 

2)  Reuse  of  generic  software  components  for  different  applications.  An 
example  of  this  would  be  for  the  Air  Force  to  reapply  radar  tracking 
algorithms  from  one  radar  system  to  another. 

?)  Reuse  of  prototype  and  development  software  during  the  evolution  of  a 
system  through  its  life  cycle.  Software  must  provide  its  appropriate 
contribution  in  support  of  DOD's  Preplanned  Product  Improvement  Ini¬ 
tiative  (PppI). 

A  new  acronym  was  established  by  the  panel  to  facilitate  discussion  and 
reporting:  RUS  is  used  herein  to  denote  Reusable  Software.  The  term 

software  package  used  herein  refers  to  a  separately  compiled  and  config¬ 
ured  unit(s)  of  computer  program  source  code. 


APPROACH 


3.0 

The  panel  initially  addressed  the  very  basic  issues  of  reusability: 
definition,  areas  of  greatest  reusability,  areas  of  least  reusability, 
and  expected  payoff.  The  following  definition  was  formed  by  the  panel: 

Reusable  software  is  existing  software,  including  specification, 
design,  code,  and/or  documentation,  which  can  be  employed  or 
adapted,  in  part  or  total,  into  a  new  end  use. 

It  was  pointed  out  by  members  of  the  panel  that  this  definition  is  in 
conflict  with  the  DAR  definition  of  software  since  the  latter  does  not 
include  specification,  design  or  docunentation  as  part  of  software.  How¬ 
ever,  for  purposes  of  the  panel  deliberations,  this  distinction  was  over¬ 
looked.  The  above  definition  does  highlight  one  specific  approach  of  the 
panel:  That  is  to  consider  reuse  of  the  specification,  and  of  the  design, 
as  well  as  the  code  itself. 

The  panel's  input  in  greatest  reusability  and  least  reusability  reflected 
the  diversity  of  background  of  the  group.  Although  there  was  no  general 
consensus,  observations  of  attributes  for  greatest  reusability  were: 
standard  interface,  structure,  and  style  of  software;  modularization  to 
very  snail  elements;  reapplication  in  a  family  of  systems;  reapplication 
by  the  same  developer;  and  applications  with  a  very  specific  single 
requirement  (e.g.,  a  compiler). 

Least  reusable  software  was  characterized  as  the  converse  of  the  above  as 
well  as  software  which  interfaces  to  sensors  (including  radar,  transduc¬ 
ers);  software  which  is  highly  machine-dependent;  and  software  which  has 
a  specific  narrow  mission.  It  was  observed  that  embedded  computer  sys¬ 
tems  generally  share  these  latter  attributes. 

The  payoff  discussion  similarly  reflected  the  diversity  of  the  group. 
Those  with  a  DOD  embedded  applications  orientation  declared  a  more  modest 
expected  payoff  (25t  cost  reduction)  than  those  from  a  business  systems 
background  (up  to  66 t)  where  reuse  has  been  a  reality  for  several  years. 
There  was  clear  consensus  that  reuse  would  improve  the  project  schedule 
and  reduce  the  risk  of  new  system  developments.  Further  benefit  would  be 
realized  throughout  the  life  cycle,  both  development  and  maintenance,  and 
would  apply  in  many  different  areas:  documentation,  testing,  training, 
reduced  personnel  skill  requirements  and  rapid  prototyping. 

Subsequent  to  these  early  deliberations,  the  panel  was  subdivided  into 
four  Subpanels: 


SUBPANEL  1: 


(Section  a. 1)  How  to  design  and  build  reusable  software. 
What  are  the  techniques,  tools  and  significant  considera- 


tions 


constructing 


software  system,  including 


specifications,  documentation  and  code,  such  that  this 
software  will  be  reusable9  The  orientation  of  this  group 


lays  the  groundwork  for  what  software  developers  should 


consider  in  order  to  achieve  reuse  in  their  future 


developments. 


SUBPANEL  2: 


SUBPANEL  3: 


SUBPANEL  4: 


the  unique  aspects  of  managing  a  project  where  at  least 
of  the  software  is  reused  instead  of  developing  all 
new  software?  The  group  considered  the  source  of  the 
software  as  either  within  one ' s  own  company  or  from  an 
outside  agency. 


(Section  4.3)  How  to  employ  existing  software  and  what 
can  be  learned  from  past  experience .  There  have  been 
projects  which  have  successfully  adopted  existing 
software.  What  were  the  lessons  (both  good  and  bad)  from 
these  efforts?  Also,  what  steps  must  be  accomplished  to 
use  existing  software? 

(Section  4.4)  Implications  on  POD  policy  and  acquisition 
practice  and  strategy.  The  group  considered  new  policy 
requirements,  impact  on  existing  standards,  and  how  to 
provide  incentives  to  government  program  managers  and 
contractors  to  reuse  existing  software.  Also,  what  role 
must  DOD  play  to  make  software  accessible  to  potential 
reusers? 


*1.0  DISCUSSION 


This  section  contains  the  reports  from  tho  four  subpanels  introduced 
above.  In  general,  it  wcs  agreed  that  a  much  greater  degree  of  reusabil¬ 
ity  in  embedded  systems  is  achievable.  Currently,  existing  software  can¬ 
not  be  expected  to  see  much  reuse  but  in  the  future,  new  software  can  be 
built  for  reuse.  However,  very  significant  changes  are  required  in 
development  methodology,  support  tools,  acquisition  practices,  and  last, 
and  most  significant,  in  the  attitudes  of  software  managers  and  develop¬ 
ers.  The  gains  in  overcoming  "private”  programming  have  only  been 
achieved  with  great  effort  and  patience.  Reuse  of  software  on  a  large 
scale  will  require  even  greater  effort  and  patience.  This  is  further 
dramatized  by  problems  of  moving  software  between  contractors.  Software 
acquisition  managers  will  have  to  recognize  that  there  are  practical  lim¬ 
itations  to  the  amount  of  reuse  which  can  be  expected  to  be  achieved  in  a 
new  application.  New  applications  always  have  some  unique  requirements, 
hence  new  code.  They  can  help  achieve  reuse  by  carefully  questioning 
each  new  requirement  to  verify  it  is  in  fact  a  requirement,  not  just  a 
"nice  to  have”  feature.  And,  finally,  by  including  design  reuse  under 
this  topic,  even  "new"  code  can  benefit  from  the  past. 


Implementing  Reusable  Software 


A  typical  software  development  cycle  begins  with  the  mapping  of  require¬ 
ments  to  software  functions,  then  performing  functional  design  of 
software,  and,  in  a  top  down  manner,  providing  a  successively  more 
detailed  design  of  software  functions,  modules  and  packages.  The  design 
"layers"  are  documented  as  specifications  (for  functions,  programs, 
etc.).  These  are  used  to  implement  program  elements  (code).  Two  end 
products  result  from  this  process,  namely  code  and  "as-built”  program 
implementation  documentation.  Figure  4.1  shows  what  is  reusable  in  the 
project  cycle:  intangible  design,  and  tangible  code,  specifications,  and 
documentation . 


Specific  techniques  for  implementation  of  reusability  are  presented  in 
Table  4.1.  The  techniques  for  generating  RUS,  which  are  described  in 
this  section,  are  generally  different  than  current  standard  practice. 

For  purposes  of  discussing  reuse  issues,  reuse  of  source  code  implies  the 
reuse  of  accompanying  program  implementation  documentation ,  specifica¬ 
tions  and  functional  design.  Reuse  of  program  specification  material 
(without  code  reuse)  also  implies  reuse  of  the  underlying  functional 
design.  Reuse  of  functional  design  alone  generally  implies  that  new  pro¬ 
gram  specifications  and  new  source  code  will  result. 

4.1.1  Functional  Design  Reuse 

Structure  and  Partitioning.  Functional  design,  to  be  reusable,  must  be 
clearly  identified,  locatable,  and  understandable.  For  maximum  benefit, 
the  design  must  be  structured  and  partitioned  such  that  individual  func¬ 
tions  become  portable  software  packages.  A  technology  of  structuring  and 
partitioning  requirements  into  functions  needs  to  be  developed.  (At 
present,  requirements  allocation  and  functional  partitioning  is  an  art 
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TABLE  4.1  RUS  TECHNIQUES/ ISSUES  AND  CONCEPTS 


Functional  Design  Reuse 

-  Structure  and  Partitioning 

-  Identification 

Requirement;.;  Fulfilled 
Generic  Function  Identification 

-  Location 

Cataloging  Scheme 

-  Understanding 

Hierarchical  Kxposition 
Language. 

Specification  and  Documentation  Reuse 

-  DID  Format 

-  Specification  Language,  Standardization 

-  Investigate  Documentation  Automatically  Generated  from  Code 

-  Machine  readable  interchange  media 

Word  processable  distribution 


Code  Reu3c 

-  Transferability  (Peoplewi.se,  assumes  no  personal  contact) 

Readability 

Package  specification 
Interface  specification 
Procedure 
Modularity 

Packaging  and  Information  Hiding 
Standards,  Conventions,  and  Style 

-  Transportation  (Systemwise) 

Language  S t an da rdi za t i on 
Toolset  Standardization 
Pnekage/Unir  Sizing 
Flexibility 

Interface  Standardization  (Intermodule  and  I/O) 
Parametrl c 

Package  Specification 
binding 
Conversion 
Task)  rig 

Timing  Independence 
Assembly  Language  Considerations 


t 


process  dependent  on  the  software  engineer's  skill.) 

Since  RUS  is  intended  to  be  put  to  a  new  end  use  after  its  initial  appli¬ 
cation,  a  number  of  special  technical  considerations  affects  its  imple¬ 
mentation.  A  future  end  use  will  not  be  known  at  the  time  of  initial 
implementation.  Judgement  must  be  exercised  in  the  partitioning  of 
requirements  and  allocation  of  functions  to  software  components  to  maxim¬ 
ize  the  probability  of  future  reuse. 

Identification.  A  unit  of  functional  design  is  identifiable  when  it  is 
described  by!  both  (1)  the  requirements  it  fulfills  and  (2)  by  the  generic 
or  specific  functions  it  implements.  As  part  of  defining  rules  and 
scientific  procedures  for  functional  design  structuring,  identification 
criteria  can  be  established.  Providing  consistent  identification  cri¬ 
teria  for  each  end  unit  of  functional  design  represents  a  new  task  for 
software  designers. 

Location .  A  unit  or  collection  of  units  of  functional  design  is  locat- 
abl’e  wFen  the  software  designer  can  readily  identify  pre-existing 
software  packages  as  candidates  for  reuse.  A  tool  providing  a  multi¬ 
dimensional  location  catalog  (index)  is  needed  to  provide  location  by 
identification  criteria  described  above.  Preparation  of  inputs  to  the 
catalog  is  performed  by  the  functional  designers  as  each  design  component 
and  software  unit  is  identified. 

Understanding.  Functional  design  is  understandable  when  it  can  be  under¬ 
stood  from  design  specifications  in  separate  design  environments  (e.g., 
different  companies,  without  personal  contact  or  communications  with  the 
original  authors) .  Design  which  fails  to  be  understandable  is  not  reus¬ 
able.  Techniques  for  go-no  go  judgment  of  design  specification  under - 
standability  need  to  be  established  and  procedurized . 

To  be  understandable,  a  top-down  hierarchical  exposition  of  the  design  is 
mandatory.  (Design  is  presently  documented  by  introductions  and  func¬ 
tional  details,  without  intervening  levels).  The  time  to  read  and  under¬ 
stand  design  communications  material  should  be  minimized  by  clear  top  and 
clear  successively  detailed  descriptions.  Figure  U.2  shows  how  the 
number  of  levels  needs  to  become  larger  than  the  two  (top  level  and  bot¬ 
tom)  presently  documented.  Reuse  of  portions  of  design  (such  as  algo¬ 
rithms  and  subprocesses)  will  be  facilitated  by  specification  material 
which  allows  the  reader  of  a  unit  of  reusable  design  to  learn  a  minimum 
of  application-specific  details  to  understand  internal  subprocesses  and 
sub functions,  other  than  those  applicable  to  the  specific  portion  which 
is  the  candidate  for  reuse.  In  the  figure ,  in  the  past  method,  to  under¬ 
stand  detail  1.2  (presumably  a  reusable  unit  candidate),  one  needs  to 
understand  interfaces,  which  are  to  1.1,  and  1.3.  Thus,  much  of  the  sys¬ 
tem  must  be  understood.  But  in  the  future  example,  to  understand 
1.1. 1.2,  one  need  only  examine  its  neighbors  (if  interfaced  to)  and  the 
higher  levels  1.1.1,  1.1,  and  1.0.  This  reduces  the  level  of  detailed 
understanding  a  future  software  user  need  gain  about  interfacing 
units/functions.  Techniques  such  as  ICAM  Definition  (IDEF)  methodology 
support  a  hierarchial  exposition  and  should  be  explored  for  wider  appli¬ 
cation  . 
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Language .  Tine  textual,  or  graphical,  means  of  functional  design  exposi¬ 
tion  is-  not  at  present  constrained.  For  reuse,  progress  toward  con¬ 
sistency  or  standardization  is  required.  Significant  advances  are  being 
made  in  the  area  of  tools  which  provide  languages  for  design  specifica¬ 
tions.  As  these  language  tools  become  standardized  and  accepted,  a 
methodology  for  their  use  for  hierarchical  functional  design  is  needed. 

4.1.2  Specification  Reuse 

While  specifications  and  documentation  are  semantically  different,  they 
have  common  attributes.  As  used  here,  specifications  describe  functional 
design,  they  "specify’'  the  implementation  of  program  code;  documentation 
describes  how  the  programs  interface  and  operate,  as  built.  Documenta¬ 
tion  is  covered  in  Section  4.1.5.  Specifications  and  documentation  need 
to  be  meaningful  conveyances  of  functional  and  detailed  design,  and  of 
software  implementation.  This  is  presently  .  difficult  to  implement  on 
large  DOD  projects.  Research  into  formats  of  design  exposition  (such  as 
hierarchical  and  multi-level  as  previously  described)  is  needed  to  iden¬ 
tify  tools  and  techniques  which  will  provide  support  to  reuse.  The 
"writing-illiteracy"  of  many  software  designers  has  caused  specification 
writing  to  be  decoupled  from  code  implementation  which  is  often  (even 
usually.’)  decoupled  from  program  description  (documenting) .  The  applica¬ 
tion  of  automated  specification  production  tools  and  graphical  computer- 
aided  design  documentation  is  needed  to  compensate  for  this  problem.  Use 
of  specification  languages  and  accompanying  tools  are  a  part  of  this 
need.  The  DOD  should  consider  building  and  providing  specification  gen¬ 
eration  and  maintenance  tools,  including  a  cannon  specification  language, 
to  contractors. 

Any  attempt  to  reuse  specifications  of  any  sort  is  presently  thwarted 
because  (unless  written  by  same  contractor,  same  department),  the  physi¬ 
cal  appearance  and  organization  of  contents  is  inconsistent.  The  present 
JLC  Monterey  I  and  Monterey  II  efforts  to  produce  a  single  set  of  Tri- 
Service  DIDS  for  software  specifications,  when  brought  to  a  completion, 
will  significantly  aid  reusability.. 

Specifications  are  not  readily  reusable  when  distributed  on  paper  (hard¬ 
copy)  .  A  DOD-wide  documentation  interchange  media  is  required  which  is 
word-processable  (for  text  and  "language")  and  graphically-processable 
(for  non-text  techniques,  as  used  com  ercially).  Procedures  and  tech¬ 
niques  for  incorporation  of  reused  specification  material  with  mechanical 
file  transfer  can  then  be  considered. 

4.1. '?  Code  Reuse 

Source  code  reuse  requires  source  code  transferability  (peoplewise,  from 
an  author,  who  is  not  in  personal  contact  with  reuser,  to  a  reuser)  and 
source  code  transportability  (physical  movement,  from  one  host/target 
combination  to  another). 

4.1. ?.l  Transferability .  Transferability  requires  human  factors  type 
considerations:  readibility,  modularity,  packaging,  information  hiding, 
and  use  of  standards,  conventions,  and  style. 
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Roadability.  Present  technology  of  languages  and  program  organization 
provides  two  basic  sections  to  a  software  package:  the  specification  and 
the  body.  The  package  specification  defines  the  naming,  calling,  and 
interface  to  the  package's  subprograms.  The  package  body  includes  inter¬ 
nal  declaration  and  procedural  code  implementing  the  packaged  sub¬ 
processes.  Both  the  package  specification  and  body  need  to  be  readable 
by  reusers,  but  for  different  reasons. 

The  package  specification,  and  in  particular  the  included  interface 
specification  sections,  needs  to  be  readable  by  a  future  software 
engineer  evaluating  the  package  for  reuse.  A  highly  underst  ndable 
description  of  algorithmic  and  control  parameters  is  critical  to  the  sui¬ 
tability  for  reuse.  The  package  specification  readability  must  address 
interfacing  to  new  and  other  reused  packages. 

The  package  body  is  not  meant  to  be  examined  by  the  average  reuser.  Tor 
DOD  system  maintainers,  the  package  body  may  be  centrally  controlled  by 
an  issuing  agency  which  may  not  even  distribute  source  code  to  using 
agencies  and  contractors.  If  distributed,  it  may  be  only  of  use  to  a 
scientific,  communications,  or  special  discipline  expert.  Thus,  the 
sophistication  of  the  code  body  may  be  higher  than  that  of  a  package 
specification. 

In  the  real  world,  even  if  the  package  body  is  intended  to  be  reused  by 
machine  code  capture  without  source  code,  access  to  its  body  portion  is 
needed  during  test,  integration,  and  maintenance.  Given  that  testing,  or 
post-deployment  retest,  uncovers  a  problem  with  the  package  operation,  a 
readable  package  body  is  necessary.  The  degree  of  sophistication  and 
knowledge  of  the  "body"  reader  will  be  higher  than  that  of  the  package 
specification  (interfacing)  reuser,  thus  allowing  sophisticated  tech¬ 
niques  of  programming  to  be  used  so  long  as  the  transferability  of  pro¬ 
cedure  understanding  can  be  conveyed  in  the  absence  of  personal  author- 
to-reuser  contact.  ' 

Modularity.  Modularity  refers  to  the  partitioning  of  a  collection  of 
functional  operations  within  a  reusable  package.  Modularity  isj  an  impor¬ 
tant  transferability  characteristic  which  encourages  the  physical  parti¬ 
tioning  of  program  code  into  small  comprehensible  units  (representing, 
for  example,  single  ideas).  Limits  on  the  number  of  lines  of  source  code 
in  a  package  enhance  the  ability  of  a  reuser  to  comprehend  its  total 
scope.  Limits  on  nunber  of  functions  performed  in  a  package  encourage 
simplicity.  Given  compact  size  and  simple  functions,  understanding  and 
reuse  is  facilitated.  Modularity  of  a  package  will  result  in  modularity 
of  the  package  specification  (the  user  desription  and  interface  section) 
and  modularity  of  the  code  internal  to  the  package  body. 

Packaging  and  Information  Hiding.  Given  functional  modularity  and  good 
partitioning  of  the  package's  internal  operations,  the  reuser  will  have 
access  to  individual  functions  as  needed.  Information  hiding  is  imple¬ 
mented  by  strict  separation  of  package  specifications  from  bodies  and  by 
judicious  unit  partitioning.  Given  that  a  package  is  a  distributable 
collection  of  software  subprograms,  careful  attention  must  be  given  by  a 
system's  designers  to  the  partitioning  of  functions  to  packages.  Like 
functions  (which  may  map  to  diverse  requirements)  should  be  candidates 


for  co-packaging.  Machine-dependent  operations  should  be  candidates  for 
mandatory  separate  packaging  from  generically  transportable  functions.  A 
formal  methodology  for  packaging  must  be  developed  in  order  for  reusable 
software  to  be  developed . 

Standards  and  Conventions.  The  issues  of  standards,  conventions  and 
style  Es  a  natural  one  to  code  transferability.  Examples  of  standards 
are:  language:  interfaces  (interprogram  and  input/output) ;  modularity; 
design  specification;  documentation;  and  programing  environment..  Stan¬ 
dardization  of  these  types  of  issues  enhances  under stand ability  of  both 
package  specifications  and  procedural  code. 

Conventions  govern  those  issues  which  cannot  be  edicted  by  a  standard. 
Examples  are:  naming  conventions,  formatting  of  listing  and  commenting. 
Universal  standards  and  conventions  will  be  required  in  order  to  transfer 
software  between  contractors.  However,  it  is  npt. feasible  to  consider 
that  all  software  developers  could  agree  on  all  standards.  Therefore, 
effort  is  required  to  isolate  those  standards  and  conventions  which  most 
effect  transferability. 

4. 1.3. 2  Transportability .  Transportability  requires  mechanical  compati¬ 
bility  from  original  system  tc  reusing  system.  Standardization  of 
language,  toolset,  and  interfaces  facilitates  transportation. 

Language  Standardization.  Language  standardization  is  an  important  con¬ 
sideration  for  code  transportability  for  both  physical  movement  of  code 
to  the  computer  of  a  new  system  and  for  the  inability  to  translate 
between  languages  (the  language  issue  is  fully  discussed  in  4.1.4). 

Toolset  Standardization.  Toolset  standardization  is  needed  so  that  a 
package's  author's  environment  is  available  to  the  reuser.  The  Ada 
effort  presently  is  experimenting  with  a  standardized  toolset,  the  Ada 
Program  Support  Environment  (APSE). 

Interface  Standardization.  This  issue  addresses  creation  of  package 
specifications  which  utilize  standardized  interfaces.  Interface  edn- 
sideratiors  are  separated  in 00  parametric  interfaces  (the  passing  of  data 
values)  4ncl  tasking  interfaces  (the  timing  independence  and  scheduling 
coordination,  e.g.Ada  "rendezvousing”)  of  sections  of  program  code. 

Parametric  interfacing  is  enhanced  by  information  hiding,  as  mentioned 
above,  as  well  as  value  format  and  parameter  order  conversion.  These 
require  use  of  HOLs  which  perform  parameter  conversion,  reordering,  and 
subroutine  acceptance  of  parameters  somewhat  independently  of  caller 
specifications.  Other  parameter  interface  issues  which  require  attention 
are  naming  conventions  and  parameter  conversion  such  as  from  fixed  to 
floating  numerics  or  scaler  to  array. 

The  reuse  of  time-dependent  software  and  the  problem  of  coordinating 
software  items  within  their  native  task  structure  environments  may  result 
in  practical  limitations  to  widespread  reuse  of  such  items.  Further 
study  is  required  on  this  point. 

Assembly  Language  Considerations.  Assembly  language  issues  cover  ISA 
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standardization,  package  separation,  and  common  data  item  reference:  ISA 
standardization  is  needed  if  reused  packages  are  to  be  allowed  to  include 
assembly  cpde.  Package  separation  is  necessary  so  that  assembly  language 
(machine-dependent  code)  is  physically  removed  from  transportable  machine 
independent  RUS. 

4.1.4  Language  Issues 

In  looking  at  the  technical  factors  which  influence  the  degree  to  which 
software  is  reusable,  it  becomes  evident  that  language  issues  must  be 
considered  as  a  primary  factor.  The  proliferation  of  languages,  dialects 
of  languages,  and  hence  language  processors,  give  rise  to  various  syntac¬ 
tic  and  semantic  differences  which  preclude,  to  a  degree,  the  reusability 
of  software.  Although  it  is  expected  that  the  transition  from  the  use  of 
existing  "standard"  languages  tc  the  use  of  Ada  will  alleviate  the  prob¬ 
lem  significantly,  the  language  problem  may  continue  to  persist  unless 
very  strict  controls  are  implemented  and  enforced  with  respect  to  both 
the  Ada  language  and  its  compiler (s).  It  is  incumbent  upon  the  responsi¬ 
ble  DOD  support  software  agency  to  take  forceful  action  to  ensure  that 
"dialects"  of  Ada  are  not  permitted  to  occur.  With  respect  to  existing 
languages  DOD  program  managers  must  take  action  to  restrict  the  use  of 
language  features  in  the  various  dialects  to  the  common  subset,  if  it 
exists. 


Similarly,  reusability  is  affected  somewhat  by  both  the  magnitude  and  the 
frequency  of  changes  which  are  applied  to  existing  languages  to  incor¬ 
porate  enhancements  or  new  features.  The  latter  is  especially  a  problem 
in  the  event  that  there  exists  multiple  dialects  of  the  same  language, 
often  managed  by  separate  agencies,  and  enhancements  are  not  incorporated 
in  each  dialect.  The  result  clearly  is  language  divergence  and,  thus, 
the  ability  to  interchange  software  elements  between  projects  using  the 
different  dialects  is  greatly  diminished.  \  \ 

i  ! 

Another  factor  which  affects  RUS  is  the  degree  jto  which  the  language  sup¬ 
ports  coding  of  required  functions  at  the  high  order  source  level.  If 
the  language  features  are  such  that  the  software  engineer  is  forced  to 
resort  to  the  frequent  use  of  assembly  language,  then  reusability  is 
affected  adversely  by  the  resultant  machine  dependencies.  It  is  incun- 
bent  upon  the  responsible  DOD  support  software  agency  to  ensure  that  the 
language  supports  the  coding  of  required  functions.  Otherwise,  and  for 
existing  languges  which  are  not  adequate,  software  developers  must  iso¬ 
late  machine  dependencies  in  order  to  achieve  reusability. 


A  point  is  frequently  made  that  the  use  of  assembly  language  is  necessary 
in  order  to  achieve  efficiency.  Hie  argument  most  often  prevails  with 
respect  to  operating  system  software  where  inefficiencies  give  rise  to 
system  overhead,  in  terms  of  both  memory  and  process^  Although  valid 
in  many  instance^,  it  is  nevertheless  the  case  that  couing  in  assembly 
language  results  in  machine  dependencies  which,  again,  tend  to  minimize 
software  reusability.  With  regard  to  this  issue,  the  DOD  Ada  Program 
Office  must  provide  compilers  which  generate  code  that  is  efficient 
enough  to  permit  the  development  of  time  critical  processes  in  HOL. 

At  the  current  rate  of  hardware  technology  advancement,  it  may  be  that 
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the  hardware  will  compensate  for  tremendous  degrees  of  inefficiency  per 
today's  standards  a  lot  sooner  than  the  compilers  can  make  very  signific- 
nat  gains  in  efficiency.  At  any  rate,  the  short  term  solution  to  this 
will  be  for  designers  to  identify  and  locate  packages  with  the  highest 
probability  of  reuse  and  ensure  those  packages  are  implemented  in  higher 
order  language. 

Another  factor  which  affects  the  rapidity  with  which  RUS  is  achieved, 
assuming  Ada  is  all  it  is  expected  to  be,  pertains  to  whether  or  not  a 
capability  is  provided  to  DOD's  program  managers  to  incrementally  upgrade 
Ada.  Since  an  enormous  amount  of  software  exists,  today  which  must  be 
supported  for  many  years  to  come,  it  might  be  worthwhile  to  consider  the 
implications  of  supporting  mixed-languages  in  the  Ada  support  environ¬ 
ment.  However  there  is  a  trade-off  of  providing  this  capability  or  hope 
that  the  other  current  standard  languages  might  "die  a  natural  death" 
much  sooner.  If  the  mixed-language  concept  is  not  supported  in  the  Ada 
environment,  and  if  we  are  not  successful  in  upgrading  existing  systems 
to  Ada,  then  the  languages  currently  in  existence  will  require  support 
well  beyond  the  year  2000. 

4.1.5  Documentation 


Documentation  requirements  in  an  environment  of  RUS  are  more  elaborate 

but  less  labor  intensive  than  with  conventional  non-reusable  software.  { 

The  techniques  to  support  the  producing  of  RUS  require  a  number  of  new 

items  of  documentation.  They  also  require  consideration  of  new  tools,  to  ] 

support  production,  distribution,  and  reuse  of  documentation.  J 

Documentation  considered  by  this  section  includes  but  is  not  limited  to:  i 

\  \  ■:] 

t  1  Functional  Design  Specifications 

Functional  Performance  Specifications  , 

Hierarchical!  Desijgn  Specifications  1 

Software  Implementation  Documentation  ^  j  j  ^ 


Program  Design  Documentation 
Code  Documentation  (Embedded  Comments) 
Interface  Design  Documentation 
Traceability  Documentation 

User  Documentation 


Testing  Documentation 


Maintenance  Docunentation 
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Tri-Service  DIDs.  A  single  set  of  documentation  formats  is  required  to 
implement  reusability  of  documentation.  A  single  responsible  agency 
(such  as  the  JLC)  will  need  to  consider  DID  evolution  in  support  of  an 
emerging  technology  for  reusability.  A  primary  reason  for  having  a  sin¬ 
gle  set  of  DIDs  is  so  that  all  services  can  reuse  packages  which  may  have 
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been  generated  for  any  other  service. 

Improved  Design  Documentation  Methodology.  Two  considerations  for 
improved  design  documentation  standards  are  hierarchical  multi-level 
design  documentation,  and  use  of  a  standardized  language  for  textual 
design  descriptions.  Each  package  specification  segment  should  have  a 
brief  summary  as  to  inputs/outputs  required,  linking  information,  who 
wrote  the  software,  and  what  it  does.  Attention  must  be  given  to  package 
specification  formats  which  are  not  subject  to  design  and  coding  technol¬ 
ogy  variations.  Thus,  material  can  be  extracted  from  one  application  and 
inserted  into  the  deliverable  documentation  of  another  without  requiring 
rewriting. 

Computerized  Tools.  A  number  of  advanced  capabilities  are  required  to 
facilitate  reuse  of  software  documentation.  These  entail  machine  distri¬ 
bution  of  word  processable  text  (including  language  prose,  if  any)  and  of 
graphically  processable  figures.  In  addition,  tools  will  provide  an 
automated  capability  to  edit  and  incorporate  revisions  to  reused  documen¬ 
tation  sections  as  they  are  issued  by  the  agencies  responsible  for  reused 
material  in  a  new  end  item. 

Documentation  Traceability.  An  absolute  essential  for  RUS  is  the  tracea¬ 
bility  of  documentation.  A  package  of  code  must  have  a  traceable,  iden¬ 
tifiable  document  action  history.  When  a  system  segment  or  package  is 
used,  documentation  segments  may  also  be  reused.  This  will  have  other 
benefits  that  are  a  requirement  for  the  ability  to  reuse  software.  If  an 
analyst  is  to  reuse  segments  or  packages  of  application  software,  he  must 
know  the  function  and  parameters  of  that  package.  Given  good  documenta¬ 
tion  and  organized  cataloging,  the  reuser  Will  know  what  is  available  and 
the  functions  the  segments  and  packages  perform. 

Two  items  of  special  interest  in  traceability  and  derivation  are  identi¬ 
fied. 

a)  Traceability  of  Requirements.  If  one  views  the  tier  of  documents  as 
successive  levels  of  requirements,  then  the  purpose  of  each  source 
statement  can  be  associated  with  a  requirement.  For  RUS,  we  need  to 
be  able  to  trace  from  system  requirements  to  a  source  statement. 
This  may  require  some  additions  to  the  documentation  standards. 

b)  Derivation  of  Family  Tree.  To  reduce  the  documentation  that  would 
result  when  a  module  is  modified  for  a  slightly  different  function, 
reusers  must  keep  a  very  precise  record  of  the  derivation  of  the 
modified  module.  Ibis  need  and  the  general  need  for  documentation  of 
the  smallest  modules  probably  mean  that  CPCJs  will  be  established  at 
lower  levels  than  normally  required. 

The  two  above  items  may  require  further  work  in  the  areas  of  CPCI  defini¬ 
tion  and  docunentation  standards. 

Automated  Test  Support  Documentation.  This  topic  covers  a  data  base  of 
automated  test  sequencing,  control ,  and  analysis  data;  the  tools  which 
automatically  test  based  on  this  data,  and  reports  produced  by  test  tools 
using  such  data.  The  test  tools  should  operate  under  much  greater 
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automated  control  then  at  present.  The  purpose  of  automated  support 
documentation  is  to  facilitate  automated  retest  of  both  the  configured 
end  item  (its  components  and  integrated  segments)  and  retest  of  the  reus¬ 
able  system  segments  delivered  to  a  RUS  library.  Automated  retest  should 
occur  automatically  upon  change  to  internally  authored  code  and  upon 
receipt  of  updates  to  reused  segments  of  code,  specification,  or  design. 

4.1.6  Tools 

Software  support  tools  are  an  important  pert  of  present-day  (1981) 
software  projects,  but  will  become  even  more  important  in  future  projects 
which  are  both  consumers  and  producers  of  RUS.  Table  4.?  characterizes 
specific  needs  for  tools  identified  in  this  section.  It  is  neither  com¬ 
plete  nor  exhaustive  but  aims  at  identifying  the  minimum  set  of  tools 
required  for  software  reuse.  As  noted  in  the  Table,  many  new  tools  will 
be  required  to  effect  an  efficient  transfer  of  software  between  develop¬ 
ers.  Investment  will  be  required  to  research  and  develop  these  tools. 

All  support  tools  must  themselves  be  transportable  and,  when  needed  to 
support  application  of  source  code  (e.g.,  preprocessors),  rehostable. 
The  languages  supported  (if  several)  must  have  the  capability  to  produce 
object  code  linkable  to  that  of  each  others  language,  and  usable  on  dif¬ 
ferent  target  machines.  Standardizing  on  a  single  language  would  elim¬ 
inate  the  need  for  language  compatibility  and  would  eliminate  the  redun¬ 
dant  effort  and  money  expended  to  maintain  and  enhance  older  incompatible 
translators.  By  restricting  the  software  development  process  to  the  tar¬ 
get  computer,  we  naturally  inherit  its  limitations  in  terms  of  both 
hardware  resources  and  support  software  tools  and,  thus,  frequently  pro¬ 
vide  to  our  software  developers  an  unsatisfactory  environment  within 
which  to  generate  software.  This  results  often  in  less  freqi\ent  recom¬ 
piles  and,  consequently,  greater  \reliance  upon  machine-dependent  code  in 
the  form  of  jpatches  to  resolve  software  errors.  The  problem  is  exag¬ 
gerated  when  source  libraries  are  not  kept  current,  when  source  libraries 
are  maintained  at  sites  remote  from  the  integration  facility,  and  when 
developers  are  not  forced  to  incrementally  recompile  programs.  Tran¬ 
sportable  standard  support  tools  capable  of  being  hosted  on  a  variety  of 
commercial  systems  will  help  to  alleviate  this  problem. 

The  concepts  of  RUS  must  be  applied  in  entirety  to  support  tools.  With 
the  increased  dependence  on  support  tools  that  reusable  application  pro¬ 
grams  will  require,  it  is  mandatory  that  support  tools  be  controlled,  and 
certified  in  a  very  thorough  manner.  The  tailoring  is  inevitable,  but  it 
must  be  controlled  such  that  it  does  not  result  in  unique  machine  or 
application  dependencies  being  inserted  into  the  application  programs. 

4.1.7  Recommendations 

There  was  a  consensus  among  this  group  that  attempts  to  reuse  present-day 
existing  software  are  likely  to  meet  with  failure.  Widespread  reuse  of 
software  will  require  that  new  software  be  developed  with  a  requirement 
that  such  new  software  be  manufactured  for  reuse.  Hence,  software 
modules,  segments,  and  items  must  be  identified  as  candidates  for  reuse 
at  their  initial  creation  based  upon  their  inherent  potential  for  reuse, 
the  expected  requirements,  and  other  economic  factors.  The  comments  of 
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TABLE  4.2 


SOFTWARE  SUPl’ORT  TOOLS  IN  REUSE  ENVIRONMENT 


1  •  Catalog  and  Source  and  recipient  of  reusable  software  (Code 

Librarian  and  Specification).  Includes  indexing  system*. 


2.  Computerized  Standardized  text  processable  and  graphically  processable 

Documentation  distribution  needed.  Used  for  functional  design*, 

program  description,  and  user  documentation.  Allows 
selected  or  complete  reuse  of  text  and  figures  from 
pre-existing  documents,  with  automatically  repeatable 
editing  and  update*  (e.g.,  so  revised  reused  specs 
can  be  automatically  reodited  and  updated). 


3.  Configuration  Tracking  and  control  of  reused  components;  tracing  of 

Management  elements  in  reusable  system  delivery  segments;  control 

of  automatic  retest*,  code  and  documentation  update 
upon  internal  or  reused  item  changes/updates. 

4.  Languages  Standardized  transportable  design  language*.  Standardized 

transportable  retargetable  programming  language. 

5.  Standard  Standardized  transportable  toolset*.  Interhost  file 

Utilities  and  media  conversion  vehicles*.  Standardized  interactive 

command  language*.  (Includes  linkers,  binders,  and 
loaders  for  reused  (object)  components.) 


6.  Verification  Specific  tools  to  enforce  standards  and  reusability, 

and  Audit 


7.  Computerized  Computer  sequence  unit  test  and  integration  testing*. 

Testing  Repeatable  automatic  retesting  upon  revision  of  reused 

packages.  Testing  both  to  end  item  configuration  and 
for  deliverable  reusable  system  segments. 


8.  System  Automated  system  generators  (for  producing  configured 

Gcnorat ion  end- item  of  software  objects). 


Denotes  tools  not  currently  available  in  present  state-of-the-art,  they  will 
require  new  development. 
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Section  4.1  would  then  form  the  methodology  by  which  the  new  software 

would  be  developed  to  achieve  the  reusability. 

Specific  recornnendations  are: 

1.  Structure  and  partitioning  methods  must  be  developed  including  func¬ 
tional  design  description  by  hierarchical  exposition. 

2.  Software  unit  packaging,  modularity  rules,  coding  standards  and 
information  binding  concepts  require  exploration  and  standardization. 

3.  Interface  standardization  criteria  (Intermodule  and  Input/Output), 
including  parametric  interface  (data  passing,  binding,  and  conver¬ 
sion)  and  tasking  interface  (timing/scheduling,  flow  of  control  and 
coordination)  require  development. 

4.  A  standard  for  identifying  functional  units  (at  the  design  level)  by 
(1)  requirement  and  (?)  generic/specific  function  is  needed.  The 
methodology  of  using  centralized  catalogs  of  reusable  software 
components/ functions  must  be  developed. 

5.  Use  of,  and  standardization  of,  specification  language  and  accompany¬ 
ing  tools  is  needed. 

6.  Develop  and  standardize  a  support  tool  environment  which  emphasizes 
transportability,  multiple  target  machine  code  generation,  and  a 
standard  support  tool  interface.  In  some  agencies,  this  is  currently 
underway  and  should  be  supported. 

7.  A  rigorous  certification  of  support  tools  and  reusable  application 
software  should  be  performed. 

8.  Require  that  support  tools  be  identified  during  the  design  phase  of 
application  software  with  consideration  given  as  to  what  tools  exist 
and  the  re-application  of  project-specific  tools  to  other  projects. 
A  controlling  board  or  standard  should  be  created  to  enforce  these 
requirements.  Project-specific  tools  which  are  needed  to  reuse  source 
code  (e.g.,  preprocessors)  must  be  transportable  and  rehostable. 

9.  A  study  should  be  initiated  to  identify  new  management  support  tools 
needed  for  control  of  the  developed  software  intended  for  reuse  such 
as  computerized  generation  and  dissemination  of  specifications, 
verifications  and  certification  of  adherence  to  reuse  standards. 

10.  For  existing  standard  languages  having  multiple  dialects,  restrict 
software  development  to  a  conrcnon  language  subset. 

11.  For  the  ultimate  set  of  standard  languages,  the  responsible  govern¬ 
ment  agency  should  maintain  strict  configuration  control  to  ensure 
that  multiple  dialects  are  not  permitted  to  occur.  Also,  the  agen¬ 
cies  should  ensure  that  coding  of  all  required  functions  is  ade¬ 
quately  supported  at  the  source  level  (without  having  to  resort  to 
direct  code) . 


12  Determine  the  feasibility  of  supporting  multiple  languages  in  the  Ada 
Support  Environment  (APSE)  and  thereby  permit  incremental  upgrade  of 
current  systems  to  the  ultimate  language  set. 

13«  Compare  maintenance  docunentation  requirements  to  reusability  docu¬ 
mentation  requirements  to  determine  what  additional  requirements 
exist. 

14.  Investigate  and  develop  traceability  methods/tools  such  that: 

a)  System  requirements  can  be  traceable  down  to  a  source  statement. 

b)  A  precise  record  of  the  module  derivation  can  be  kept  (parent¬ 
offspring  relationships) . 

4.2  Managing  a  "Ported"  Development 

The  purpose  of  this  subpanel  was  to  identify  the  differences  that  exist 
between  the  management  of  a  development  project  employing  some  amount  of 
reusable  software  and  the  management  of  a  development  project  employing 
no  RUS.  All  areas  of  project  management  that  required  special  emphasis 
were  identified.  In  the  subpanel,  all  project  management  functions  were 
analyzed,  and  it  was  determined  that  there  were  differences  or  special 
emphasis  required  in  the  following  project  management  areas: 

o  Project  organization 
o  Project  control 
o  Configuration  management 
o  Quality  assurance 
o  Testing 
o  Documentation 
o  Library 

Three  conditions  of  RUS  were  considered.  They  are  as  follow: 

1.  The  new  development  project  reuses  software  developed 
within  it's  own  company. 

2.  The  new  development  project  reuses  software  supplied 
by  the  Government  as  GFE. 

3.  The  new  development  project  reuses  software  developed 
and  resident  within  a  company  other  than  the  company 
using  it. 

4.2  1  Reusable  Software  Organization  (RUSO) 

Within  each  system  or  software  producing  entity,  both  industry  and 
government,  an  organizational  structure  must  be  established  which  specif¬ 
ically  addresses  the  reuse  of  software.  The  objectives  of  the  RUSO  arc 
to  establish  an  effective  company-wide  reuse  program.  The  goals  are: 
reduced  development  time;  reduced  duplication  of  effort;  improved  system 
reliability;  and  reduced  costs.  An  effective  RUS  program  will  ensure 
that  productivity  on  software  development  projects  is  improved  ond  the 
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cost  of  software  is  reduced.  The  effective  RUS  program  will  also  cause 
generated  software  components  to  be  likely  candidates  for  future  reuse. 

Initially,  the  RUSO  should  review  current  software  development  practices 
and  determine  where  standardization  for  reusability  would  have  the 
greatest  payoff  in  future  reuses.  In  the  beginning,  software  packages 
containing  procedural  routines  usually  offer  the  highest,  payoff  for  stan¬ 
dardization  efforts. 

Upon  identifying  areas  for  standardization,  the  RUSO  should  evaluate  sup¬ 
port  software  in  use  for  its  appropriateness  as  a  "standard  development 
environment".  In  the  likely  event  adequate  development  environment  tools 
do  not  exist  to  support  reuse,  the  RUSO  shall  undertake  to  procure, 
develop,  and  otherwise  install  the  required  tools. 

As  the  RUSO  matures  in  its  function,  serious  consideration  should  be 
given  to  developing  applications  software  components  in  anticipation  of 
future  systems/ programs  needs.  This  requires  that  the  RUSO  coordinate 
with  business  planning  and  marketing  groups  to  identify  areas  in  which  to 
concentrate  the  building  of  reusable  components. 

The  system  documentation  of  designated  tools  should  be  maintained  by 
RUSO.  Such  a  practice  helps  to  ensure  software  maintenance  in  the  event 
original  software  authors  depart  the  organization.  The  documentation  of 
reusable  components,  usually  in  the  form  of  the  user  guides,  will  be 
critical  to  the  overall  success  of  the  RUSO.  The  documentation  will  be 
the  primary  mechanism  for  transferring  reusable  software  throughout  the 
programing  community.  The  RUSO  should  employ  a  professional  technical 
writer  to  ensure  that  the  documentation  quality  is  of  a  level  that  will 
be  readily  accepted  by  the  organizations'  programming  community. 

The  effectiveness  of  the  RUSO  program  should  be  measured  on  a  routine 
basis  to  ensure  that  the  RUSO  is  meeting  pre-established  goals.  Produc¬ 
tivity  measurements  may  be  obained  by  developing  baseline  estimates  on 
the  amount  of  time  required  for  the  various  stages  and  functions  of  an 
application  development.  For  example, 
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Baseline  estimates  for  comparison  would  be  from  past  performance  on  prior 
contracts  for  each  specific  application.  A  specific  set  of  baseline 
estimates  must  be  prepared  for  each  discrete  application  stage  or  func¬ 
tion.  All  future  projects  should  be  monitored  for  their  performance 
against  these  baselines  to  determine  the  productivity  gains. 
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4.2.2  Project  Control 


The  actual  cost  of  employing  RUS  on  a  new  development  is  valuable  infor¬ 
mation  to  have  from  both  an  historical  perspective  and  from  a  software 
cost  control  perspective.  It  is  anticipated  that  there  will  be  a  cost 
savings  by  employing  RUS,  however,  the  actual  savings  is  not  known  at 
this  time.  The  Project  Manager  of  the  new  development requires  up-to- 
date  information  on  each  RUS  component  used  so  that  he  can  make  informed 
continuing  estimates  of  project  cost  and  schedule.  Additionally,  from  a 
corporation  or  government  standpoint,  it  is  very  useful  to  have  a  history 
of  the  actual  costs  of  various  RUS  elements. 

With  these  requirements  in  mind,  it  becomes  imperative  that  the  RUS  to  be 
used  in  the  new  development  be  included  in  the  WBS  in  such  a  way  as  to 
allow  the  tracking  and  collection  of  RUS  costs.  For  instance,  each  piece 
of  RUS  should  be  identified  as  a  separate  work  package.  Thus,  historical 
information  will  be  gathered  which  could  be  used  to  estimate  future  costs 
and  cost  savings. 

u.2.3  Configuration  Management 

Prior  to  the  start  of  any  project  using  RUS,  the  corporate  CM  office  and 
the  project  CM  office  must  prepare  for  the  RUS  environment.  Existing  CM 
plans,  policies,  and  procedures  must  be  revised  to  accommodate  the 
employment  of  RUS.  This  effort  will  be  accomplished  in  coordination  with 
the  RUSO  in  addition  to  normal  coordinating  efforts  such  as  with  other 
company  offices. 

The  RUSC  should  establish  a  separate  and  distinct  library  for  the  RUS 
(see  Section  4.2.7).  Procedures  and  controls  must  be  established,  as 
described  in  4.1,  such  that  anyone  can  access  the  contents  of  the  library 
sufficiently  to  determine  if  there  are  library  elements  which  would  be  of 
use  to  the  new  development.  Hie  library  also  addresses  recapture  of  the 
new  software  components  developed  for  reuse  on  future  projects.  Of 
paramount  importance  in  the  library  is  ease  of  access  by  future  develop¬ 
ment  projects,  as  covered  in  4.1.1,  under  "Location". 

The  RUS  library  components  should  be  identified  by  the  referenced  loca¬ 
tion  scheme.  Changes  made  to  an  RUS  component  will  cause  the  changed 
element  to  be  re-identified.  This  facilitates  automatic  retest  of  sys¬ 
tems  employing  the  changed  component. 

RUS  supplied  to  a  contractor  by  the  Government  would  be  identified  by  the 
RUSO  using  the  same  identification/location  scheme  used  to  identify  other 
GFE. 

The  prime  responsibility  for  configuration  control  rests  with  the  RUSO. 
This  is  accomplished  by  establishment  of  a  RUS  Change  Control  Board 
(RCCB)  whose  charter  woul^  be  to  maintain  the  integrity  of  the  RUS  and 
its  library.  Any  change  required  to  be  made  to  RUS  by  a  project  employ¬ 
ing  the  RUS  must  be  submitted  to  the  RCCB  in  the  form  of  a  change  propo¬ 
sal  prior  to  the  change  being  made.  The  RCCB  is  the  sole  authority  for 
the  approval,  disapproval,  or  deferral  of  proposed  changes.  Figure  4.7 
identifies  the  change  processing  flow  for  RUS  changes  required  and 
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proposed  by  the  project  employing  the  RUS.  If  the  change  to  be  imple¬ 
mented  is  to  be  made  to  every  project  employing  the  RUS,  the  RUSO  will 
accomplish  the  change  and  submit  the  new  version  to  the  users.  If  the 
change  to  be  implemented  is  to  be  made  only  to  the  single  project  RUS, 
the  project  will  implement  the  change  and  submit  a  copy  of  the  changed 
RUS  to  the  RUS  library.  The  RUSO  should  validate  the  change  and  then 
incorporate  it  into  the  library. 

It  should  be  noted  that  if  a  project  proposed  change  is  disapproved  by 
the  RUSO,  the  project  can  still  implement  the  change  but  the  RUSO  will 
probably  then  not  continue  to  support  the  RUS. 

4.2.4  Quality  Assurance 

Software  Quality  Assurance  (SQA)  functions  remain  largely  the  seme  in  the 
RUS  environment.  However,  additional  plans,  policy,  and  procedures  may 
be  required  to  accommodate  certain  differences.  The  company  SQA  office, 
in  coordination  with  the  RUSO,  CM,  and  the  software  department,  all 
should  assist  in  establishing  SQA  criteria  and  attributes  which  will 
qualify  software  to  be  identified  in  RUS. 

The  SQA  Group  should  review,  evaluate,  and  verify  the  RUS  package  for 
completeness  and  compliance  with  the  company-established  SQA  criteria  and 
attributes  for  RUS  prior  to  placement  of  any  elements  into  the  library. 
Upon  receipt  of  an  RUS  package  from  a  source  external  to  the  company,  the 
SQA  Group  should  participate  in  the  evaluation  of  the  RUS  package  for 
completeness  and  performance  as  stated. 

4.2.5  Testing 

One  of  the  benefits  of  employing  RUS  should  be  reduced  component-level 
testing.  Ihe  degree  of  reduced  testing  will  vary  depending  on  several 
factors.  The  source  from  which  the  RUS  comes  appears  to  be  one  factor 
which  will  affect  the  amount  of  testing  conducted.  If  received  from  the 
government,  the  RUS  is  more  likely  to  have  been  subjected  to  rigorous 
testing  and  therefore  will  likely  require  less  testing  by  the  recipient. 

The  documentation  and  tests  available  to  the  RUSC  should  include  a  com¬ 
plete  test  package  from  the  development  organization  which  initially  pro¬ 
duced  the  RUS.  This  package  should  include  the  RUS  test  plans,  pro¬ 
cedures,  results  and  reports.  It  may  also  include  test  programs  and 
drivers,  test  scenario,  etc.  This  allows  the  reuser  to  assure  that  the 
tests  cover  the  intended  scenario  of  operation. 

In  the  case  in  which  the  project  company  receives  the  RUS  from  another 
company,  additional  testing  of  the  RUS  is  probably  required  to  establish 
a  performance  benchmark  or  system  baseline.  The  test  group  will  probably 
have  a  bigger  task  in  adapting  and  integrating  the  other  company's  RUS 
test  plans,  procedures,  etc.,  into  the  project  test  plan  and  procedures. 
The  size  of  the  task  will  depend  upon  the  amount  and  quality  of  test 
documentation  received  in  the  RUS  package.  Inter-company  standardization 
of  test  plans  and  procedures  would  help  alleviate  this  expense. 

The  approach  to  formal  testing  of  RUS  should  be  that  of  testing  the  RUS 
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at  the  highest  level  of  testing  first.  T.f  the  RUS  portion  of  the  project 
software  fails  at  the  higher  level  of  testing,  then  regressive  testing 
would  be  conducted  on  the  RUS  to  isolate  the  erroneous  elements. 

4.?. 6  Documentation 

A  key  responsibility  of  the  RUSO  must  be  to  provide  documentation  for  the 
RUS  which  is  suitable  for  adapting  to  each  project.  Thus,  a  standard  set 
of  documents  must  be  provided  for  each  RUS  element.  The  documents  should 
be  n  superset  of  those  normally  required  by  the  companies'  customer  base. 
Tncl tided  in  the  documentation  should  be  an  abstract  designed  to  quickly 
or iafit  the  potential  RUS  user  to  the  specific  element.  This  documenta¬ 
tion  required  by  the  RllSO  with  each  RUS  element  should  be  determined  and 
identified  in  RUS  standards  and  procedures  developed  by  the  RUSO.  Sec¬ 
tion  4.1  discussed  specific  documentation  practices  to  promote  RUS.  When 
the  RUS  is  changed  because  of  project  requirements,  the  documentation 
must  be  submitted  to  the  library  along  with  the  copy  of  the  changed  code. 

When  providing  RUS  to  a  contractor,  the  government  has  the  responsibility 
to  provide  comprehensive  docunentation  with  the  software  elements.  As 
updates  are  made  by  the  government's  original  source,  the  documentation 
on  these  changes  must  be  passed  on  as  well. 

NOTE:  In  general,  whenever  the  government  puts  themselves  into  the  posi¬ 
tion  of  providing  software  to  contractors,  they  must  assume  the  responsi¬ 
bility  for  on-going  baseline  control  and  dissemination  of  that  same 
software  to  every  re-use  application. 

4.2.7  Reusable  Software  Library 

A  RUS  Library  must  be  established  by  the  RUSO  to  retain  master  copies  of 
each  piece  of  the  RUS  along  with  copies  of  the  necessary  docunentation. 
As  each  RUS  is  changed,  the  library  will  ensure  that  updated  docunenta¬ 
tion  and  updated  masters  replace  the  existing  masters  in  the  library. 
Additionally,  the  library  must  ensure  that  a  historical  record  of  the  RUS 
is  kept. 

The  RUS  Library  is  responsible  for  the  distribution  and  dissemination  of 
RUS  required  by  individual  projects.  Additionally,  as  RUS  is  revised  by 
the  RUSO,  the  revised  RUS  and  revised  docunentation  is  distributed  to 
projects  employing  the  RUS. 

•One  of  the  tasks  that  the  RUS  Library  must  accomplish  is  the  development 
of  a  system  for  storage,  numbering,  indexing,  etc.,  to  ensure  not  only  an 
organized  filing  system  but  also  an  organized  and  efficient  method  of 
retrieval.  The  RUS  Library  will  index  the  PUS  stored  in  the  library. 
The  library  should  periodically  distribute  throughout  the  company  the 
index  of  RUS  to  ensure  that  company  personnel  are  aware  of  the  RUS 
library  contents. 

4.2.8  Recommendations 

The  following  specific  recommendations  resulted  from  this  subpanel : 
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1.  A  Reusable  Software  Organization  should  be  established  within  each 
large  organizational  entity,  to  effect  the  reuse  of  software.  Their 
responsibilities  should  include  standards  and  methods  for  developing 
reusable  software,  enforcing  documentation  standards  and  controlling 
the  reusable  software.  Tn  addition,  the  RUSO  should  be  concerned 
with  performance  measurement  to  assure  the  reuse  applications  are 
cost  effective. 

2.  Project  control  mechanisms  must  be  adjusted  to  account  for  RUS. 
Specifically,  the  WBS  must  allow  for  visibility  into  the  RUS. 

3.  Configuration  management  for  RUS  must  be  centralized  with  the  RUSO. 
As  changes  are  needed,  they  should  be  individually  approved  and 
incorporated  by  the  central  organization,  disseminated  to  all  users, 
and  accounted  for  at  distinct  changes.  All  of  the  RUS  must  be 
retained  within  a  central  master  library  administered  by  the  RUSO. 
The  library  would  be  responsible  for  control  and  dissemination  of  all 
packages  and  for  historical  records  related  to  reuse  of  the  software. 

4.  Testing  and  documentation  methods  should  be  examined  in  RUS  applica¬ 
tions.  Automated  regression  tests  and  high  level  testing  have  an 
increased  importance.  A  standard  modular  set  of  documents  must  be 
available  for  each  RUS  package.  Testing  documentation  needs  to  be 
portable  along  with  the  software  itself. 

4.3  Learning  from  Past  Experience 

Over  the  years,  some  members  of  the  commercial  business  sector  have  had 
excellent  success  in  utilizing  the  reusability  technique  in  the  develop¬ 
ment  and  maintenance  of  business  application  software.  This  section 
examines  the  lessons  learned  from  these  experiences;  addresses  contrac¬ 
tual  considerations  for  the  planning,  award  and  performance  of  the  system 
and  software  contracts;  looks  at  documentation  needs  and  user  groups;  and 
provides  recommendations  to  the  DOD  community  for  achieving  greater  reuse 
of  software. 

A  case  study  of  the  reuse  of  business  application  software  is  presented 
in  Appendix  E.  In  brief,  this  specific  experience  is  as  follows: 

During  the  last  five  years,  one  division  of  a  large  company 
developed,  implemented  and  perfected  a  reusable  design  and  code 
technique  supported  by  software  packages,  tools  and  services.  The 
concept  behind  this  methodology  was  to  accelerate  application 
development  through  the  elimination  of  redundancy  in  the  software 
cycle.  Techniques  of  functional  modularity  are  employed  to  prepare 
modules  for  use  in  multiple  applications.  These  modules  are 
designed,  coded,  tested,  documented,  certified,  and  stored  in  a 
library  for  later  use.  To  date,  a  central  library,  containing  over 
2500  certified  reusable  modules  composed  of  76,000  lines  of  code, 
has  been  built.  The  payback  for  this  one  time  investment  is  close 
to  a  million  lines  of  reusable  code.  This  technique  has  largely 
been  responsible  for  the  development  of  eight  new  data  base  manage¬ 
ment  applications  with  an  average  of  60  percent  reusable  code.  In 
addition,  the  methodology  imposes  a  top-down  approach  to  develop,  it 
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in  which  high-level  logic  is  designed  and  coded  first.  The  result 
is  the  production  of  logic  structures,  prewritten  for  each  of  six 
types  of  business  application  programs.  Each  logic  structure  serves 
as  a  "working  shell"  to  be  completed  by  reusable  modules  and  unique 
code.  Over  ^000  new  programs  have  averaged  *1?  percent  reusable  code 
through  the  use  of  logic  structures.  By  applying  these  techniques, 
it  was  found  that  40  to  60  percent  of  the  redundancy  in  business 
application  development  can  be  eliminated. 

This  implementation  experience  could  serve  as  a  logical  model  for  future 
reusability  programs  for  the  DOD.  Most  of  the  experiences  described 
below  come  from  this  successful  program  and  from  information  exchanges 
among  businesses  which  have  implemented  or  are  contemplating  implementing 
reusability  programs. 

4.3.1  RUS  Development  Philosophy  Lessons 

The  implementation  of  RUS  methodology  forms  a  system  to  develop  systems. 
Because  it  is  a  system  itself,  the  RUS  methodology  should  pass  through 
the  same  development  activities  as  any  other  system  -  from  identifying 
the  project  goals  to  installing  the  system.  The  main  project  goal  for 
the  development  of  an  RUS  methodology  is  to  formally  integrate  structured 
technology  (analysis,  design,  coding,  testing,  and  documentation  tech¬ 
niques)  into  a  set  of  practical  guidelines  -  a  system,  if  you  will,  for 
developing  automated  software  systems. 

Experience  in  implementing  these  structured  techniques  has  indicated  that 
these  guidelines  must  be  used  in  order  for  the  reusability  concept  to 
work  within  an  organization.  It  is  difficult,  if  not  impossible,  to 
create  and  reuse  software  unless  structured  technology  is  utilized.  How¬ 
ever,  experience  has  also  shown  that  some  business  systems  analysts  and 
programmers  find  it  difficult  to  adapt  to  structured  technology  princi¬ 
ples.  The  reason  behind  this  problem  is  that  their  past  training  and 
experience  have  conditioned  them  to  think  in  a  procedural  fashion  rather 
than  in  a  hierarchical,  functional  way.  One  solution  is  to  identify  and 
train  personnel  who  can  adapt  to  this  kind  of  structured  environment  and 
then  use  them  in  a  pilot  project.  Another  strategy  is  to  institute  an 
entry-level  training  program.  The  program  should  concentrate  heavily  on 
structured  and  reusability  techniques.  It  was  found  that  an  entry-level 
training  program  is  one  of  the  best  investments  an  organization  can  make 
when  trying  to  overcome  the  inherent  problems  of  using  structured  tech¬ 
nology  and  reusability  concepts. 

In  sum,  the  RUS  methodology  used  to  standardize  system  development 
activities  should  impose  a  structured,  disciplined  approach  on  the 
development  process  and  enhance  communicetior.  between  all  members  of  the 
project  team  and  organization. 

4.3.2  Installation  Considerations 

More  important  than  the  formulation  of  project  goals  is  the  proper 
installation  of  RUS  methodology.  (If  a  system  is  not  correctly 
installed,  then  it  may  not  be  used  at  all*)  There  are  four  major  factors 
to  consider  when  installing  RUS  methodology.  Listed  in  order  of 
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importance,  they  are:  political/psychological  factors;  installation  and 
training;  the  training  medium;  ongoing  support  and  monitoring;  and  con¬ 
sulting.  In  the  following  paragraphs,  these  considerations  are  discussed 
briefly. 

Political  and  Psychological  Factors.  These  factors  can  profoundly  affect 
user T s  acceptance  of  RUS  methodology.  The  most  important  consideration 
is  the  conmitment  by  high  level  management  to  the  installation  of  RUS 
methodology.  In  order  to  enlist  their  backing,  managers  must  be  educated 
in  the  advantages  of  using  a  new  methodology.  It  is  important  that  the 
methodology  be  adopted  throughout  the  organization,  because  isolated 
groups  using  the  methodology,  or  part  of  it,  will  nt  be  able  to  communi¬ 
cate  effectively. 

The  second  consideration  is  the  potential  feeling  of  loss  of  freedom  or 
of  power,  as  well  as  the  natural  resistance  to  change,  on  the  part  of  new 
users  of  RUS  methodology.  To  combat  this  resistance,  the  methodology 
must  appear  simple  to  use.  This,  of  course,  depends  on  the  training, 
tools,  and  services  (such  as  an  automated  data  dictionary)  provided  to 
support  the  methodology.  The  quantity  of  forms  and  control  procedures 
that  a  company  assigns  to  go  along  with  the  methodology  will  also  affect 
its  complexity.  It's  best  to  keep  procedures  to  a  minimum  until  person¬ 
nel  become  accustomed  to  the  methodology. 

Finally,  there  is  the  possibility  that  users*  acceptance  of  RUS  methodol¬ 
ogy  will  be  extreme,  and  that  the  methodology  will  be  taken  too 
literally.  Some  data  processing  personnel  may  look  upon  it  as  a  "cook¬ 
book"  solution  to  all  their  problems,  instead  of  as  an  aid  to  their  own 
thought  processes.  This  could  potentially  lead  to  the  problem  of  the 
developers  supporting  the  methodology,  as  opposed  to  the  methodology  sup¬ 
porting  the  development  process. 

Aware  of  these  considerations,  the  following  steps  can  be  taken  to  minim¬ 
ize  their  effect: 

o  Classify  the  methodology  as  a  guideline  rather  than  as  a  standard 
and  allow  users  to  customize  it  whenever  problems  are  found  in 
its  use,  thereby  allowing  the  methodology  to  be  viewed  as  a  help, 
not  as  a  restriction. 

o  Set  up  a  review  group  consisting  of  managers  whose  departments 
will  use  the  methodology,  thus  giving  them  ownership  and  a  part 
in  the  decision  to  accept  the  methodology  (beware,  however,  that 
reluctant  managers  may  make  any  meeting  of  this  group  their  pol¬ 
itical  battleground). 

o  Prove  that  the  methodology  works  by  offering  statistics  from 
other  users  of  the  methodology  or  results  of  a  pilot  project. 

o  Send  key  personnel  to  reusability  seminars  and  conferences.  For 
example,  the  IBM  Guide  User’s  Conference  has  a  project  on  reusa¬ 
bility. 

o  Carefully  consider  methods  of  training  new  users  to  apply  the 
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Installation  and  Training.  There  are  four  general  strategies  that  can  be 
used  to  install  a  methodology  and  to  train  its  eventual  users. 

o  The  Prototype  Project  Strategy  -  The  installation  is  tested  by 
applying  the  techniques  on  a  reasonably-sized  prototype  project, 
and  the  resulting  documentation  from  the  project  is  used  as  sup¬ 
port  for  adoption  of  the  methodology.  Tine  developers  of  the  pro¬ 
totype  project  are  then  used  to  assist  or  even  train  members  of 
other  projects.  Experience  has  indicated  that  this  usually  is 
the  best  approach  for  gaining  acceptance  of  the  reusability  con¬ 
cept  in  most  organizations. 

o  The  Walk-Before-You-Can-Run  Strategy  -  Training  in  the  methodol¬ 
ogy  consists  of  teaching  only  those  techniques  that  will  be  used 
immediately  by  members  of  new  projects.  For  example,  a  course  in 
structured  design  techniques  is  held  before  structured  design 
activities;  a  reusable  design  course  before  reusable  design 
activities,  and  so  on.  This  training  is  repeated  for  each  new 
project  team. 

o  The  Functional  Job  Description  Strategy  -  Training  courses  are 
held  for  each  category  of  employee  (analyst,  designer,  coder, 
database  developer),  starting  with  the  course  applicable  to  the 
specific  role  and  proceeding  to  courses  on  activities  with  which 
the  employee  may  need  to  interface.  A  programmer,  for  instance, 
would  take  a  course  in  reusable  coding  techniques  first,  and  a 
design  overview  course  later.  This  is  a  realistic  strategy  for 
groups  that  are  functionally  organized. 

o  The  Educate-the-Masses  Strategy  -  In  this  approach,  all  available 
staff  members  ere  trained  when  a  course  in  any  of  the  techniques 
is  scheduled,  regardless  of  whether  they  will  need  to  use  the 
techniques  in  the  near  future.  This  strategy  also  advocates 
training  every  employee  in  all  of  the  techniques  at  once.  The 
approach  is  usually  not  successful  unless  the  personnel  can  apply 
the  techniques  on  a  project  at  the  end  of  the  training. 

As  with  any  system  development  effort,  a  plan  should  be  developed  to  mon¬ 
itor  and  control  the  installation  of  RUS  methodology  and  its  associated 
training.  Realistically,  a  combination  of  the  above  strategies  will  be 
needed  for  most  (large)  environments. 

The  following  list  denotes  some  training  requirements  for  each  level  of 
participant: 

o  High-Level  Management  -  Requires  an  overview  of  the  reusability 
concept . 

o  Middle  Management  -  Requires  an  overview  and  possibly  an  intro¬ 
duction  to  the  structured  techniques 

o  Project  Leaders  -  Require  a  working  knowledge  of  the  complete 


methodology . 

o  Technical  Project  Staff  Members  -  Require  a  detailed  working 
knowledge  of  the  activities  for  which  they  are  responsible  and 
possibly  an  overview  of  other  activities. 


Training  Medium.  How  RUS  methodology  is  taught  will  greatly  affect  its 
understand ability  and,  therefore,  its  acceptability.  Although  cost  will 
probably  determine  the  training  medium  used,  some  options  include  a 
workshop  course,  a  lecture  course,  and  a  self-study  audio-visual  course. 

A  workshop  course  is  one  of  the  best  forms  of  training,  as  it  provides 
the  staff  with  hands-on  experience  in  applying  a  technique  with  immediate 
feedback  to  questions.  Experience  has  demonstrated  that  an  entry-level 
workshop  training  program  can  provide  the  foundation  required  to  imple¬ 
ment  a  successful  reusability  program.  Personnel  graduating  from  an 
entry-level  training  program  can  then  be  assigned  to  experienced  person¬ 
nel  who  will  believe  in  reusability  techiques.  This  approach  ha3  worked 
out  extremely  well  and  is  cost  effective.  A  lecture  course  also  provides 
immediate  answers  to  questions  and  is  usually  easier  and  cheaper  to 
present,  but  doesn't  offer  participants  the  chance  to  practice  using  the 
techniques.  A  self-study  audio-visual  course  allows  easy  assimilation  of 
material  and  is  a  user-friendly  mediun. 

All  of  these  media  can  be  backed  up  with  assistance  from  outside  consul¬ 
tants  and  on-the-job  advice  of  experienced  personnel.  When  using  this 
approach,  the  best  results  come  from  using  a  combination  of  training 
vehicles,  such  as  a  workshop  course  with  follow-up  visits  from  the 
consultant/advisor . 

On-Going  Support  and  Monitoring.  RUS  methodology,  like  any  system, 
should  be  maintained  Eo  keep  it  from  becoming  out-of-date.  A  support 
group  or  possibly  one  person  must  be  responsible  for  this  task.  The 
importance  of  the  task  should  not  be  minimized  for  the  data  processing 
industry  is  developing  rapidly;  new  software  development  and  support 
techniques  are  being  introduced  in  virtually  every  phase.  For  the  metho¬ 
dology  to  survive,  it  needs  to  accommodate  pertinent  innovations,  but  the 
process  of  incorporating  them  must  be  controlled  according  to  the  same 
principles  used  in  the  original  methodology.  This  rule  applies  to  any 
customizing  of  the  methodology,  either  for  the  organization  as  a  whole  or 
for  a  particular  project.  The  effectiveness  of  RUS  methodology  should  be 
tracked  and  compared  with  results  using  the  old  technique  to  determine 
the  cost-effectiveness  of  the  methodology.  Based  on  these  results, 
adjustments  to  the  methodology  should  be  made  when  necessary.  Conse¬ 
quently,  measurement  data  must  be  collected  before  total  installation  of 
the  methodology  has  taken  place. 

Consulting  -  Finally,  a  point  to  stress  is  the  need  for  support  consult¬ 
ing"!  Sn  expert  in  the  methodology  (someone  who  has  used  the  techniques, 
in  a  project  or  in  the  original  training  group)  should  be  available  to 
help  out  on  new  projects  to  ensure  that  the  methodology  is  being  used 
correctly.  This  can  be  accomplished  simply  be  having  the  advisor  acces¬ 
sible  during  reviews  or  walkthroughs. 
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In  stmnary,  the  installation  (and  on-going  supports  of  any  system 
requires  a  great  commitment  of  cost  and  effort.  A  RUS  methodology  is  no 
exception. 

H. 3. 3  Contractual  Considerations 

The  reusability  of  software  has  an  impact  on  the  planning,  award,  and 
performance  of  system  and  software  contracts.  The  following  are  some  of 
the  existing  or  anticipated  problems  having  contractual  implications r 

0  Adequacy  of  Documentation  -  A  meaningful  competitive  proposal  cannot 
be  prepared  or  evaluated  without  the  availability  of  thorough  docu¬ 
mentation  of  the  software  which  must  or  may  be  reused.  This  is  true 
whether  the  reuse  of  software  is  by  the  government  in  the  Request  for 
Proposal  (RFP)  or  initiated  by  the  contractor  in  a  proposal  to  gain  0 
competitive  edge.  If  initiated  by  the  government,  0  complete  docu¬ 
mentation  package  must  be  mace  available  during  the  solicitation 
period  and  consideration  should  be  given  to  extending  this  period  to 
allow  a  thorough  examination  by  potential  bidders.  The  RFP  should 
also  indicate  that  if  a  bidder  initiates  the  reuse  of  software,  a 
complete  docunentation  package  must  be  made  available  for  the  evalua¬ 
tion  and  any  proprietary  rights  clearly  identified.  The  documenta¬ 
tion  package  must  include  detailed  test  procedures  and  reports. 

o  Contractual  Direction  -  Problems  arise  in  performing  a  contract  reus¬ 
ing  government-furnished  software  if  the  contract  does  not  specify 
the  rules  of  how  it  is  to  be  used.  It  must  be  made  clear  if  reuse  of 
the  software  is  mandatory  or  at  the  discretion  of  the  contractor.  It 
also  must  be  made  clear  whether  the  software,  if  used,  can  be  modi¬ 
fied  by  the  contractor  with  or  without  formal  government  approval  and 
participation  in  Configuration  Control  Doard  activities. 

o  Liability/Responsibility  -  The  area  of  liability  or  responsibility 
must  be  clearly  addressed  in  the  contract  if  software  will  be  pro¬ 
vided  by  the  government  either  by  direction  or  at  the  request  of  the 
contractor.  Contract  requirements  without  a  means  to  measure  compli¬ 
ance,  usually  cannot  be  enforced  or  lead  to  government-contractor 
conflict. 

o  Incentives  -  New  technologies  and  development  procedures  are  best 
encouraged  by  providing  a  profit  motive  for  their  introduction. 
Incentive  fees  in  contracts  may  be  a  workable  mechanism  for  this,  but 
care  must  be  exercised  to  preclude  arbitrary  reuse  of  code  without 
proper  regard  for  cost  effective  benefits  to  the  government.  How¬ 
ever  ,  an  incentive  plan  must  not  overlook  the  need  for  government 
investment  in  software  tools,  aids,  services,  training  and  library 
development  for  the  enhancement  of  reusability.  Investment  in  these 
areas  should  be  included  in  the  DOD  Computer  Resources  Technology 
Plan.  Other  methods  of  funding  experiments  in  the  reuse  of  software 
should  be  explored  by  the  JLC-JPCG-CRM.  These  include  the  cost-plus 
contracts  where  reuse  is  required  or  parallel  engineering  develop¬ 
ments  where  reuse  is  specified  in  one  development  and  new  design  in 
the  second. 
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4.3.4  Docunentation 


"Code  breakage”  between  prototype  systems  and  first-item  production  units 
has  demonstrated  some  necessary  but  not  sufficient  requirements  for  code 
reusability.  Changing  requirements,  even  though  minor,  significantly 
hamper  the  use  of  prototype  code  in  a  production  item.  This  difficulty 
is  further  complicated  if  the  original  code  is  not  in  an  HOL  and  is 
poorly  docunented.  Although  this  ’’breakage”  may  be  acceptable  in  the 
prototyping  environment,  when  code  is  being  developed  with  the  intent 
that  it  is  to  be  reusable,  the  requirements  for  programming  in  an  HOL 
with  proper  and  thorough  docunentation  are  essential.  JLC-JPCG-CRM 
should  continue  to  support  the  development  and  use  of  HOLs  and  the  defin¬ 
ition  of  appropriate  documentation  standards. 

4.3.5  User  Groups 

Specific  previous  experience  in  transferring  both  hardware  and  software 
technology  has  provided  some  insight  into  the  necessary  elements  of  an 
environment  for  reusability.  History  has  demonstrated  that  transfer  of 
technology  can  be  accomplished  by  the  establishment  of  User  Groups.  In 
the  area  of  software,  many  User  Groups  exist  such  as  DECUS  and  SHARE/7. 
Software  exchange  and  reuse  frequently  take  place  between  individuals  in 
these  groups.  Necessary  ingredients  of  this  transfer  are  the  meeting  of 
individuals  working  on  similar  problems  in  similar  environments  with 
follow-up  technical  support.  In  the  examples  mentioned  above,  the 
follow-up  support  is  informal  and  normally  carried  on  by  telephone  or 
computer  network  communications.  JLC-JPCG-CRM  should  sponsor  the  forma¬ 
tion  of  I'ser  Groups  in  the  software  areas  which  they  consider  candidates 
for  cost-effective  reusability.  NOTE:  The  IBM  Guide  Conference  User 
Group  is  currently  sponsoring  a  reusable  design  and  code  productivity 
technique  project. 

4.3.6  Recommendations 

The  following  specific  recommendations  resulted  from  this  subpanel: 

1.  The  CRM  should  define  a  technology  transfer  program  which  incor¬ 
porates  lessons  learned  in  business  applications  to  Weapon  System 
Programs.  Included  would  be  a  pilot  project  to  gain  the  necessary 
experience  needed  to  implement  reusability  on  a  global  level.  This 
demonstration  project  could  also  be  used  as  a  logical  model  for  DOD 
environments  similar  to  the  business  model  referenced  in  Appendix  B. 

?..  JLC  should  provide  a  government  investment  plan  for  reusable  software 
technology  development.  Included  should  be  incentives  to  invest  IR&D 
and  overhead  ii*  reusable  software  technology  and  tool  develop¬ 

ment,  respectively.  Also,  training  programs  and  an  organizational 
structure  for  their  implementation.  A  plan  for  library  support  ser¬ 
vices  on  a  DOD-wide  basis  should  be  addressed. 

3.  JLC  should  determine  the  feasibility  (possibly  under  contract)  of 
standardizing  on  a  language  and  methodology  for  system  and  software 
requirement  analyses.  This  study  should  identify  applications  that 
are  made  amenable  to  the  use  of  requirements  analysis  methodologi  :-s 
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and  evaluate  the  possibility  of  DOD  adopting  a  single  methodology  or 
methodologies  for  various  applications.  The  study  would  also  address 
the  impact  of  adopting  a  methodology  on  government/contractor  rela¬ 
tionships  and  contractor  performance  on  DOD  contracts. 

4.  Existing  DOD  guidebooks  should  be  updated  to  address  technical  prob¬ 
lems  and  remedies  in  the  planning,  award  and  performance  of  contracts 
where  the  reuse  of  software  is  anticipated.  Standards  end  acceptance 
criteria  must  be  developed  to  determine  compliance  with  contract 
requirements  to  reuse  software  or  to  generate  reusable  software. 

5.  CRM  should  sponsor  the  formation  of  User  Groups  in  areas  considered 
candidates  for  reusability. 

4.4  Impact  or  DOD  Policy  and  Acquisition  Practice 

The  use  of  digital  computers  continues  to  pervade  military  systems.  A 
major  limitation  to  realizing  the  full  potential  of  this  technology  is 
the  time,  cost  and  risk  of  software  acquisition.  The  concept  of  reusable 
software  holds  the  dual  benefits  of  significantly  reducing  the  develop¬ 
ment  time  and  life  cycle  cost  of  military  systems,  as  well  as  increasing 
the  labor  pool  available  for  development,  operations,  and  maintenance  of 
computer  software.  A  DOD  Software  Reusability  Program  should  include  the 
following  goals: 

a)  Reduce  software  implementation  personnel  requirements. 

b)  Reduce  the  maintenance  contribution  to  life  cycle  costs. 

c)  Reduce  the  elapsed  time  from  identification  of  an  operational 
requirement  to  system  deployment. 

d)  Reduce  the  variety  and  level  of  skills  required  for  system  operation 
and  maintenance. 

e)  Reduce  overall  life  cycle  risks  and  costs. 

Achieving  reuse  requires  new  thinking  in  DOD  policy  and  acquisition  prac¬ 
tice  as  outined  in  the  following  paragraphs. 

4.11.1  Augmentation  of  DOD  Policy 

DOD  policy  on  software  acquisition  must  reflect  a  desire  for,  and  a  com¬ 
mitment  to,  software  reusability.  This  concept  should  then  be  made  a 
requirement  by  integrating  the  various  facets  of  reusability  into  exist¬ 
ing  directives.  DODD  *000.29  should  be  changed  to  promulgate  DOD's  pol¬ 
icy  on  software  reusability.  Next  the  DSARC  Guidebook  should  be  updated 
to  include  questions  of  program  managers  on  the  various  areas  of  reusa¬ 
bility  (i.e.,  can  this  requirement  be  satisfied  with  existing  software? 
Will  this  new  software  have  application  in  other,  later  developments, 
etc.?).  Also  required  is  a  DOD  Instruction  5000. RUS  which  would  be  a 
companion  '  5000.31  and  5000. 5X,  stating  DOD’s  requirements  on  reuse. 

The  indivi  _  services  must  then  promulgate  their  own  implementing 
instructions. 


The  DOD  acquisition  process  must  contain  the  appropriate  incentives  for 
getting  both  program  managers  and  contractors  to  carry  out  the  intent  of 
the  official  policy.  They  must  be  convinced  that  to  develop  RUS,  and  to 
include  in  their  projects  the  reuse  of  existing  software  i3  in  their  best 
interests.  In  the  case  of  government  program  managers,  the  DSARC  lever¬ 
age  cited  above  will  be  effective  only  in  a  limited  sense;  they  must  also 
be  enlightened  and  convinced  that  RUS  will  provide  them  with  very 
specific  benefits  during  the  acquisition  cycle  of  their  system.  Items 
like  lower  risk,  reduced  costs  and  reduced  development  time  are  examples 
of  the  payoffs  that  they  are  most  interested  in.  Contracts  can  be  incen- 
tivised  through  the  source  selection  and  award  fee  processes.  Proposal 
evaluation  criteria  should  contain  weighting  factors  that  reward  offerers 
for  capturing  existing  software.  This  requirement  should  then  be 
integrated  into  the  government's  monitoring,  review  and  auditng  pro¬ 
cedures  to  ensure  continued  compliance.  Finally,  an  award  fee  should 
contain  a  factor  for  the  amount  of  software  actually  reused  (this  can  be 
directly  measured). 

To  effectively  implement  the  DOD  policy  on  RUS,  action  must  be  taken  to 
modify,  where  appropriate,  the  various  contractual  vehicles  that  can 
influence  the  various  aspects  of  reusability.  The  following  is  a  partial 
list  of  contract  mechanisms  that  must  be  analyzed  for  impact  and  then 
changed  accordingly  to  support  the  concept. 

o  Defense  Acquisition  Regulations 

o  Military  Standards  &  Specifications 

o  Statement  of  Work  Preparation 

o  Work  Breakdown  Structure  Preparation 

o  Solicitation  Process 

o  Proposal  Evaluation  Criteria 

o  Source  Selection  Process 

o  Contract/Project  Incentives 

It  is  imperative  to  involve  the  industry  in  implementing  the  concept  of 
RUS  throughout  the  POD.  They  must  be  frequently  briefed  on  the  status  of 
DOD  initiatives  in  the  area.  Their  review  and  comments  must  be  solicited 
in  order  to  capitalize  on  their  experience  and  expertise.  All  documents 
promulgated  should  be  coordinated  with  the  industry  through  the  various 
associations  such  as  ETA,  FDPA,  NSIA  and  ATA. 

The  government's  thinking  regarding  the  specification  of  contract 
deliverables  must  be  revised.  When  a  contractor  is  tasked  to  develop  a 
specific  item  of  software  that  will  ultimately  be  reusable,  that  item 
must  be  documented,  designed,  tested,  etc.,  to  a  greater  degree.  This 
must  be  incorporated  in  Project  Plans,  must  be  budgeted  for,  and  QA  and 
CM  must  be  augmented. 

The  DOD  should  coordinate  their  efforts  with  other  organizations,  both 
nationally  and  internationally,  that  potentially  have  similar  or  related 
interests.  Candidates  are:  ANSI,  NBS,  Foreign  Military  Sales  Offices, 
NATO,  ISO. 

Areas  of  Necessary  Standardization 


If  software  reusability  is  to  be  a  viable  objective,  DOD  efforts  to  pro¬ 
mote  and  promulgate  standardization  will  have  to  be  expanded  beyond  the 
current  areas  of  high  order  language  (HOL)  and  instruction  set  architec¬ 
tures  (ISA),  to  the  entire  programming  and  operational  support  environ¬ 
ment.  The  primary  areas  of  additional  standardization  deal  with  the 
interfaces  that  any  RUS  element  must  have  with  the  overall  software  sys¬ 
tem. 

High  Order  Language.  Reusability  at  the  source  code  level  makes  the  use 
of a  standard  flOL  an  absolute  necessity.  In  this  light,  the  on-going  DOD 
activities  to  restrict  the  proliferation  of  programing  languages  and, 
further,  to  ultimately  arrive  at  a  single,  common  high  order  language 
(i.e.,  Ada)  is  the  necessary  prerequisite  for  making  RUS  a  reality. 

Instruction  Set  Architecture.  DOD  activities  directed  toward  standardi¬ 
zation  of  computer  instruction  set  architectures  can  also  be  viewed  as 
supportive  of  the  concept  of  RUS.  The  benefits  derived  from  standardiza¬ 
tion  at  this  level  do  not  apply  as  much  to  reusability  of  applications 
software  since  reusability  at  the  object  code  level  tends  to  be  less 
viable  than  reusability  at  source  code,  design,  or  algorithm  levels. 
Rather,  instruction  set  architecture  standardization  supports  reusability 
of  support  software  tools  that  are  applied  in  the  development  process. 

Operating  System/Executive  Inter faces .  The  concept  of  RUS  would  be 
greatly  enhanced  if  the  Dd)D  standardization  activities  could  be  extended 
to  address  the  idea  of  a  standard  operating  system  or  executive.  How¬ 
ever,  in  light  of  the  variety  of  operating  onvionments  and  consequent 
demands  imposed  on  the  operating  systems  and/or  executives  to  function  in 
those  environments,  a  standard  operating  system  is  probably  infeasible. 
It  is  feasible,  however,  to  consider  standardizing  the  way  in  winch  an 
element  of  software  interfaces  with  the  operating  system  (e.g.,  the  way  a 
package  is  invoked  by  the  operating  system) .  Standardization  at  this 
level  would  eliminate  the  need  to  modify  an  element  of  software  to 
achieve  compatibility  with  the  operating  system  or  executive  being  used 
for  a  given  application. 

Data  Base  Interfaces.  For  reasons  similar  to  those  stated  for  standar¬ 
dizing  IKe  operating  system  interface,  the  way  that  software  elements 
store  and  retrieve  data  from  a  global  data  base  should  be  standardized. 
Data  handling  provisions  of  Ad?  will  aid  in  this  area,  however,  if  sub¬ 
stantial  modification  to  achieve  data  base  compatibility  is  to  be 
avoided,  additional  standardization  is  required. 

Inter-Function  Interfaces.  The  operating  system  and  data  base  interface 
standardization  activities  'described  above,  coupled  with  the  procedure 
definition  provisions  of  Ada  will,  in  substantial  measure,  standardize 
the  functional  inter-relationships  among  elements  of  software.  Whether 
in  fact,  these  prove  to  be  sufficient  t.o  effect  this  standardization  is 
unclear  at  present.  This  is  an  area  for  additional  study. 

Inter-Computer/External  Interfaces .  An  obvious  candidate  for  DOD  stan- 
dardization  activity  is  in  digital  communications.  Some  work  has  already 
been  done  in  this  area  (e.g.,  MIL-STD-155?:  data  busses).  This  has  pri¬ 
marily  addressed  avionics  applications  to  date.  This  activity  needs  to 
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be  broadened  to  encompass  other  forms  of  digital  communications  for 
applications  such  as  CCCI  and  fire  control  systems. 

4.4.?  Additional  General  Comnents  Regarding  Practices 

A  nunber  of  additional  issues  loom  as  significant  in  addition  to  the 
above  two  major  areas.  These  ares  support  requirements,  data  collection 
and  dissemination,  configuration  management,  and  promoting  acceptance. 

Support  Requirements.  For  software  to  be  effectively  reused,  the  support 
requirements  must  5e  examined.  Software  development  practices  such  as 
modern  programming  techniques  must  be  reviewed  to  determine  what  the 
lower-level  elements  will  look  like.  The  over  twenty  Integrated  Logistic 
Support  Plan  items  will  have  to  be  tailored  to  support  reuse.  Quality 
Assurance  programs  will  have  to  be  expanded  to  audit  for  reusability. 
For  example,  several  (i.e.,  correctness,  maintainability,  efficiency, 
testability)  of  the  eleven  commonly  applied  quality  factors  will  be 
directly  impacted  by  RUS.  Reliablity  measurements  that  also  insure  reu¬ 
sability  must  be  developed. 

Data  Collection/Distribution .  Tn  order  to  make  RUS  a  reality  on  anything 
Hit  an  intra-company  basis,"  it  will  be  necessary  to  estabish  a  mechanism 
for  the  accumulation,  maintenance  and  distribution  of  RUS.  If  done  prop¬ 
erly,  this  RUS  library,  implemented  at  the  DOD  level,  would  serve  as  a 
repository  for  descriptive  data  for  all  software  identified  as  being 
potentially  reusable  together  with  a  pointer  identifying  the  location  of 
the  code,  documentation  and  test  data  for  each  software  element.  A  data 
storage  retrieval  system  needs  to  be  established  with  data  retrievable  on 
a  key-word  or  relational  basis  to  respond  to  contractor  or  program  office 
requests  for  existing  software  that  is  potentially  applicable  to  a  new 
program. 

An  alternative  approach  would  be  to  establish  a  RUS  library  within  each 
service  to  exchange  top-level  information  (key-word  lists,  software  ele¬ 
ment  names,  etc.)  and  to  provide  a  cross-reference  to  the  other  services' 
library  lists.  Docunentation  will  assume  a  more  important  role;  without 
adequate  documentation,  reuse  will  be  limited  to  applications  within  the 
same  company.  Greater  emphasis  on  tri-service  standards  and  documenta¬ 
tion  contents  will  be  required. 

Configuration  Management.  While  the  concept  of  RUS  has  definite  confi¬ 
guration  management  implications,  configuration  management  need  not  have 
direct  impact  on  the  establishment  or  maintenance  of  the  RUS  library.  It 
will,  however,  be  necessary  to  devise  appropriate  procedures  for  keeping 
the  library  updated  with  data  reflecting  current  versions  of  the 
software.  Also,  procedures  to  maintain  current  status  once  the  software 
has  been  imbedded  in  multiple  new  applications  are  required.  Reuse  will 
affect  configuration  management  practices  as  to  the  allocation  of  CPCIs 
and  level  of  control . 

Promoting  Acceptance.  The  introduction  of  the  RUS  program  to  the  acquis¬ 
ition  process  must  be  multifaceted.  The  concept  is  often  not  an  accepted 
practice  for  personnel  currently  involved  in  computer  software  design, 
implementation,  integration,  and  test.  Resistance  to  change  to  the 
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status  quo  is  likely.  Education  is  the  primary  means  to  help  balance  the 
perception  of  risk  and  benefit  occasioned  by  a  RUS  program.  Methods  of 
education  should  include  familiarization  in  service  schools  and  briefings 
to  both  government  and  industry.  "Road  shows"  of  both  high  and  low 
detail  as  well  as  "guidebooks"  should  be  used  to  build  a  consensus  for 
software  reusability.  Audiences  should  include  sponsors,  project 
managers,  source  selection  authorities,  industry  leaders,  and  contracting 
personnel.  Training  curricula  must  be  revised  to  promote  adaption  of 
existing  software  and  group  collaboration  rather  than  individual  effort 
and  zero  base  approaches  to  programing  projects. 

Dual  incentives  are  needed  for  both  industry  and  government  to  both  pro¬ 
duce  reusable  software  and  capitalize  on  existing  software  when  designing 
new  systems . 

4.4.4  Recommendations 

The  transition  to  maximize  the  amount  of  RUS  will  not  be  easy.  The 
implied  significant  cost  and  manpower  savings  gives  DOD  a  strong  motiva¬ 
tion.  However,  this  does  not  motivate  the  individual  program  manager  or 
contractor.  To  be  effective,  changes  must  be  made  in  the  basic  way  the 
government  does  business.  Policy  must  be  updated,  standardization  fos¬ 
tered  and  contractual  vehicles  updated.  Education  is  important;  if 
government  and  industry  recognize  the  potential  of  RUS,  the  mechanics  and 
implementation  will  follow.  How  fast  will  be  determined  by  the  accep¬ 
tance  and  emphasis  by  top  level  management,  government,  and  contractor 
personnel . 

The  following  specific  recommendations  are  offered: 

1 .  Update  the  following  documents  to  reflect  DOD  policy  and  emphasis  on 
reuse  of  software:  DODD  5000.29,  DSARC  Guidebook,  and  specific  con¬ 
tractual  vehicles.  In  addition,  the  individual  services  must  issue 
supporting  implementing  instructions. 

?.  Promulgate  a  new  DODI  5000. RUS  on  Software  Reusability. 

3*  Develop/provide  incentives  for  DOD  PM’s  and  contractors  to  support 
the  concept.  Oie  necessary  element  of  this  is  to  establish  an  educa¬ 
tional  program  to  make  both  government  and  industry  aware  of  the 
potential  benefits.  A  guidebook  detailing  practices  might  also  be  of 
benefit. 

4.  Re-evaluate  deliverable  requirements  to  be  consistent  with  reuse  by 
organizations  different  than  the  developer. 

5.  Re-evaluate  support  requirements  for  reusability,  to  include  develop¬ 
ment,  ILSP  requirements,  Q.A.,  reliability  and  configuration  manage¬ 
ment  practices. 

6.  Conduct  a  study  to  identify  and  define  a  feasible  approach  for  reus¬ 
able  software  data  collection  and  distribution. 

7.  Expand  DOD  standardization  activities  to  include  standard  software 
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package  interface  techniques  (operating  system,  data  base,  and  exter¬ 
nal  conmunication) . 

8.  Training  curricula  in  DOD  for  W's  and  in  industry  for  software 
designers  should  be  revised  to  promote  reusable  software. 

9.  Legal  questions  concerning  reused  software  as  it  relates  to  competi¬ 
tion  must  be  examined. 
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5.0 


RECOMMENDATIONS 


The  panel  deliberations  resulted  in  a  large  number  of  recommendations  for 
subsequent  action  and  a  nunber  of  areas  requiring  special  attention  by 
DOD  and/or  industry.  These  are  all  reported  in  Section  M.O.  The  major 
recorrmendations,  have  been  extracted  and  are  presented  here  to  consoli¬ 
date  the  conclusions  of  Panel  E. 

1.  Standards,  instructions,  directives,  guidebooks,  training  criteria, 
regulations,  and  contractual  vehicles  must  be  revised  to  identify  the 
changes  which  must  be  made  to  include  the  concept  of  reusability. 
The  JLC  should  foster  preparation  of  new  policy  documents  to 
encourage  and  enforce  reusability. 

2.  There  must  be  provisions  within  both  industry  and  government  for 
focusing  on  the  establishment  of  3  reusable  software  (RUS)  program. 
cArresponding  investments  in  new  techniques,  tools  and  training 
should  be  applied  by  both  government  and  industry.  A  dedicated  Reus¬ 
able  Software  Organization  may  be  required  within  each  company  to 
standardize  and  control  software  designated  for  reuse. 

3.  Tncentives  should  be  provided  for  DOD  PM' s  and  contractors  for  com¬ 
pliance  with  reusability  concepts. 

U.  Reassessment  of  the  deliverables  currently  specified  in  DOD  contracts 
must  be  based  on  added  information  needed  to  effect  reuse  of  the 
software.  Additional  data  collection  and  distribution  of  information 
will  need  explicit  attention.  Support  requirements  must  also  be  re¬ 
evaluated  for  applicability  (e.g.,  QA,  CM,  ILSP)  in  a  reuse  environ¬ 
ment. 

5.  The  success  of  RUS  in  business  applications  has  been  much  greater 
than  that  of  embedded  computer  systems.  A  more  thorough  analysis  of 
the  management,  techniques,  and  tools  used  to  achieve  this  will  yield 
valuable  insight  to  individuals  operating  in  the  DOD  envirorcnent. 
Pilot  projects  in  proven  reuse  techniques  should  be  considered  to 
establish  a  DOD  methodology. 

6.  The  JLC  should  foster  investigation  into  methods  for  effecting  the 
transfer  of  software  between  agencies  and  contractors.  This  may 
include  library  and  information  access  techniques  available  to  all 
interested  parties  and  user  groups. 

7.  Standards  and  methods  for  requirements  analysis  and  formal  specifica¬ 
tion  languages  should  be  pursued. 

8.  Functional  design  techniques  must  be  defined  to  focus  on  identifica¬ 
tion,  structure  and  partition  for  reuse.  The  design  must  be 
cataloged  and  be  accessible  independent  of  the  code.  Modularity 
guideliens  must  be  reviewed  for  appropriate  packaging  and  package 
specifications  for  reuse. 

9.  Ne  oit;n  and  coding  techniques  for  interface  standards  will  need 
attention.  Binding  between  components  and  with  operating 
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systems/executives,  convertibility  for  moving  to  new  architectures, 
avoidance  or  control  of  timing  and  sequence  relationships  are,  all 
examples  of  these  areas.  DOD  or  system  family-wide  stand;  rds  may  be 
required.  Further,  coding  standards  with  greater  emphasis  on  readi- 
bility,  individual  style  restrictions,  built-in  flexibility  for 
change,  and  parametric  bindings  are  needed. 

10.  Research  and  development  of  new  support  tools  specifically  to  promote 
reusability  are  needed.  Such  new  areas  include  specification  genera¬ 
tion  and  dissemination,  verification,  and  certification  of  reused 
components,  library  methods  and  tools  for  cataloging  and  access  of 
software  components,  and  management  tools  for  project  control. 

11.  Development  and  standardization  of  a  support  tool  environment  must  be 
initiated  to  emphasize  transportability,  address  multiple  target 
machines  and  consider  standard  support  tool  interfaces.  Support 
tools  must  be  rigorously  validated  and  certified.  In  some  agencies, 
this  is  currently  underway  and  should  be  supported. 

12.  Exclusive  use  of  higher  order  language  is  a  single  very  strong  neces¬ 
sary  precedent  for  reuse.  To  the  degree  that  multiple  dialects  (sub¬ 
sets  or  supersets)  are  allowed,  reusability  will  be  adversely 
affected . 

13.  Further  attention  is  required  to  the  transition  to  Ada  from  current 
languages.  A  large  investment  in  CMS-2,  FORTRAN  and  JOVIAL  programs 
will  not  be  discarded  for  many  years  to  come.  One  consideration 
would  be  to  have  the  Ada  PSE  support  multiple  languages.  Further, 
some  languages  (e.g.,  Atlas,  COPOL)  may  not  be  replaced  by  Ada;  these 
languages  will  need  continued  attention  to  ensure  that  all  required 
functions  can  be  implemented  in  the  higher  order  language. 

1H.  Special  investigation  is  required  into  current  documentation  stan¬ 
dards  to  achieve  reusability.  This  includes  allowing  for  a  clear 
traceability  of  requirements  through  the  entire  document  hierarchy 
(so  when  accessing  a  reusable  component,  the  associated  documents  are 
easily  identified  and  extracted).  Also,  investigation  is  required  of 
the  relationship  between  CPCI's  and  reusable  units  (i.e.,  CKII's  may 
be  defined  to  a  lower  level  than  at  present  to  facilitate  reuse) . 

Brief  consideration  was  given  to  microprocessor  software  by  the  panel. 
Hardware-intensive  applications  would  not  be  areas  for  consideration  for 
design  for  reuse.  Software-intensive  applications  might  be  candidates 
for  RUS,  however,  these  applications  tend  toward  machine-dependence  by 
plan.  Hence,  attempts  to  apply  reuse  principles  to  non-complex  micropro¬ 
cessor  software  should  be  carefuly  weighed  to  ascertain  if  it  is  economi¬ 
cally  advisable. 


Risks.  All  new  endeavors  generally  have  risks  associated  with  them.  A 
large  scale  effort  to  reuse  software  in  new  applications  is  no  exception. 
In  the  process  of  deliberation,  Panel  E  did  identify  certain  risks  atten¬ 
dant  to  developing  and  reapplying  software  in  new  applications.  These 
are  summarized  below. 


o  Hardware  trends  (VL?T,  VHSIC''  are  moving  towards  standalone  hardware 
devices  which  are  quite  capable  and  have  distinct  functional  capabili¬ 
ties.  The  world  of  reuse  in  the  future  may  center  around  configuring 
systems  with  such  functional  hardware  elements.  As  such,  the  software 
interfaces  will  become  hardware  interfaces  and  software  reuse  would 
require  adjustment  accordingly. 

o  If  software  is  to  be  reused  after  the  initial  development,  consider¬ 
able  attention  must  be  paid  to  the  proper  control  and  support  of  the 
fielded  software.  That  is  to  say,  once  the  development  is  done,  the 
software  has  transitioned  to  a  maintenance  environment.  If  someone 
else  is  going  to  extract  the  software  from  that  maintenance  environ¬ 
ment  for  reuse,  the  maintenance  staff  must  be  very  careful  to  make 
sure  that  all  of  the  development  documentation  is  in  fact  sustained 
and  all  of  the  development  tests  are  available  to  the  reuser.  This 
corrment  would  apply  whether  the  reusable  software  is  being  provided  by 
DOD  or  is  just  being  retained  within  the  organization  of  a  single  con¬ 
tractor  . 

o  Reusability  must  be  addressed  as  an  objective  of  the  development  pro¬ 
cess.  One  cannot  build  software  and  then  decide  (after  the  fact)  they 
intend  to  reuse  it.  At  the  time  the  development  is  started,  specific 
guidelines  for  methods  and  standards  must  be  established  to  support 
the  intended  reuse. 

o  A  corollary  to  this  is  that  not  all  software  will  be  able  to  be 
reused.  Certain  designs  will  not  lend  themselves  to  new  applications. 
Thus,  even  though  software  has  been  developed  with  reuse  as  an  objec¬ 
tive,  if  the  application  is  different  enough  or  the  interfaces  are 
different  enough,  the  reuse  may  not  become  a  reality.  There  has  to  be 
a  sensitivity  to  such  incompatibility;  reuse  cannot  be  arbitrarily 
forced . 

o  A  certain  amount  of  generality  must  be  injected  to  the  software  to 
support  its  reuse.  For  example,  standard  interfaces  and  standard  pro¬ 
gramming  languages  readily  come  to  mind.  This  generality  may 
translate  into  inefficiency  of  operation  and  limitations  of  accuracy. 
When  one  goes  to  the  next  application,  if  the  inefficiency  or  inaccu¬ 
racy  is  intolerable,  then  the  reuse  has  been  lost.  Thus,  in  order  to 
achieve  the  reuse,  some  considerations  may  be  required  to  allow  for 
less  than  total  optimum  performance  of  a  new  application. 

o  In  the  short  term  of  this  Panel's  deliberation,  there  was  not  an  ade¬ 
quate  dialog  as  to  the  cost  attendant  with  the  development  of  reusable 
software.  How  much  more  would  it  cost  to  develop  software  which  is 
intended  for  reuse  than  to  develop  it  for  single  application?  There 
definitely  is  an  increased  cost  and  a  specific  return  on  investment 
curve  could  be  plotted.  One  must  be  careful  to  ascertain  that  the 
cost  reduction  in  the  new  application  will  compensate  for  the 
increased  cost  of  the  original  development. 

o  Legal  (or  quasi-legal)  questions  relative  to  the  impact  of  reuse  prac¬ 
tices  on  contract  competition  might  present  some  concern  to  DOD. 
Practices  must  be  addressed  to  both  present  software  (or  allow  access 
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to  software)  to  contractors  when  reuse  is  an  expected  part  of  the  pro¬ 
curement;  and  also  to  assure  no  single  contractor  receives  unfair 
preferential  treatment  in  procurements  where  reuse  is  expected. 
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A  SUCCESSFUL  APPROACH  TO  THE  REUSE  OF  BUSINESS 
APPLICATION  SOFTWARE:  REUSABILITY  MODEL 


INTRODUCTION 


Picture  yourself  with  the  following  problem. 

• 

You  are  the  Systems  and  Programming  Manager  responsible 
for  eight  Programmer  Analysts.  You*  prime  respor.  ;ility 
is  maintaining  a  large  dock-to-stock  manufacturing  system 
with  90  to  100  terminals.  The  system  runs  on  a  Ur.ivac  1106. 
You  are  using  a  data  management  system  developed  by  your 
company.  Your  management,  for  economy  and  centralization 
reasons  tells  you  to  rewrite  the  system  for  an  IBM/ 370, 
located  25  miles  from  your  plant.  The  system  will  use 
IBM's  Information  Management  System,  You  and  your  people 
know  nothing  about  this  new  environment.  However,  one 
of  your  eight  Programmer-Analysts  has  had  previous  exper¬ 
ience  in  a  IBM  0/S  environment.  You  are  given  four  months 
to  rewrite  the  system.  At  the  end  of  this  time,  your 
Univac  1106  will  be  removed.  There  is  one  positive  note, 
however,  your  eight  Programmer-Analysts  have  a  good  grasp 
of  the  present  system.  What  development  approach  do  you 
use  under  these  circumstances? 

Needless  to  say,  a  project  with  these  conditions  han  all 
the  earmarks  for  disaster  and  under  the  circumstances  your 
best  bet  would  be  to  get  your  resume  polished  up  and  move 
on. 


In  1975,  a  group  of  eight  Programmer  Analysts  in  our  Raytheon 
Missile  Systems  Divisidn  located  in  Bedford,  Massachusetts 
did  not  polish  up  their  resumes.  Instead,  faced  with  these 
conditions,  they  carried  structured  techniques  to  their  next 
logical  step,  reusable  cobol  source  copy  code. 

These  eight  dedicated  believers  in  reusable  code  techniques 
developed  sixty-two  programs  with  an  average  of  65%  reusable 
source  lines  of  code  and  over  300  reusable  COBOL  source  code 
modules.  The  system  was  completed  on  time  with  minimum 
testing  and  impact  on  our  user  community  and  has  run  with 
minimum  maintenance  and  problems  over  the  last  four  years. 
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Due  to  this  remarkable  achievement!  a  new  era  in  software 
development  began  in  Raytheon  Missile  Systems  Division  in 
April,  1976.  At  that  time,  our  business  data  processing 
management  initiated  a  three  man,  four  month  study  to 
determine  how  to  implement  this  reusable  code  technique  in 
three  plants  and  to  make  recommendations  for  additional 
software  tools  and  aids  to  increase  our  programming 
productivity. 

OUR  ENVIRONMENT 


The  environment  studied  was  a  business  applications  environ* 
ment  using  IBM's  ANSI  COBOL  with  full  extensions  and,. where 
applicable,  IBM's  Information  Management  System.  The  work 
force  is  comprised  of  approximately  120  Programmer  Analysts 
located  in  three  different  physical  locations  with  three 
different  Data  Processing  Managers. 

STUDY  APPROACH 

Our  approach  to  the  problem  of  HOW  to  globalize  this  reusable 
code  concept  was  to  determine  what  type  of  programming  work 
was  produced  in  each  of  the  three  locations,  what  tools  were 
required  to  improve  this  work,  what  kind  of  training  was 
required,  how  much  training  was  required  and,  how  to  measure 
and  monitor  our  progress. 

After  examining  work  records  for  a  two  year  period  and  inter¬ 
viewing  three  Managers,  twelve  Supervisors,  and  other  key 
personnel,  we  determined  the  following  about  our  work  profile, 
personnel  and  programming  environment. 

SYSTEMS  PROGRAMMING  AND  ADMINISTRATION  WORK  PROFILE  FINDINGS 

To  evaluate  the  type  of  worV  our  170  people  performed,  the 
study  team  developed  a  work  profile  that  consisted  of  eight 
categories  of  work. 

1.  NEW  DEVELOPMENT  -  The  development  of  a  new  application 
not  presently  on  the  computer. 

2.  REPAIR  -  The  fixing  of  operational  bugs. 

3.  ENHANCEMENT  -  The  addition  or  modification  of  code  or 
program  to  an  existing  computerized  application. 

4.  CONVERSION  -  The  transformation  of  an  existing  computer- 
ized  application  from  one  language  to  another  or  from 
one  computer  to  another . 
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5.  REPLACEMENT  -  The  development  of  a  computer  application 
that  td’.'aa  the  place  of  one  already  existing. 

6.  SYSTEMS  SUPPORT  -  Manpower  dedicated  to  total  user 
support  oi  a  given  application. 

7.  CONTRACT  MANPOWER  -  People  not  tied  to  any  specific 
application  but  supplied  by  the  hour  to  do  what  is 
required  by  the  contracting  organization. 

8.  OVERHEAD  ACTIVITIES  -  Information  systems  manager  and 
staff  engaged  in  administrative  and  support  activities. 

A  percentage  work  breakdown  for  the  above  categories  was 

as  follows: 


1. 

New  Development 

15.0 

2. 

Repair 

7.5 

3. 

Enhancement 

30.3 

4. 

Conversion 

9.7 

5. 

Replacement 

14.5 

6. 

System  Support 

6.7 

7. 

Contract  Manpower 

8.3 

8. 

Overhead/ Administrative  Activities 

8.0 

100.0 

The  above  profile  showed  that  15%  of  our  effort  over  a  two 
year  period  was  expended  on  new  application  development* 

8%  on  overhead  and  77%  on  what  we  defined  as  maintenance. 

PERSONNEL  SKILLS  STUDY  FINDINGS 


To  determine  the  work  skills  of  our  personnel*  a  general 
skills  inventory  questionaire  was  completed  py  our  120 
programmer/analysts.  The  results  were  as  follows: 

1.  3%  had  received  formal  training  in  program  design. 
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?.  12%  had  received  formal  training  in  systems  analysis. 

3.  Less  than  10%  had  received  formal  training  in  testing 
and  documentation  techniques.; 

The  same  study  was  conducted  in  three  other  large  companies 
with  close  to  the  same  results. 

As  a  result  of  this  study,  it  was  apparent  that  formal  train¬ 
ing  and  assistance  after  training  would  be  mandatory  if  the 
REUSABLE  CODE  approach  was  to  work. 

OUR  PROGRAMMING  ENVIRONMENT  FINDINGS 


Approximately  1200  new  programs  were  written  annually 
and  they  were  categorized  Into  three  groups: 

1.  New  application  development  programs  for 
new  systems. 

2.  One-time  programs  written  for  our  user 
community  which  required  quick  response, 
usually  less  than  one  week. 

3.  Programs  written  to  enhance,  fix,  convert 
or  replace  existing  systems  which  also 
required  quick  response,  usually  less  than 
one  week. 

In  addition  to  our  new  program  findings,  we  determined 
that  we  enhance  or  fix  approximately  2000  programs  annually. 

After  analyzing  this  data  from  our  programming  work  environ¬ 
ment,  we  arrived  at  the  following  conclusions: 

1.  Approximately  37%  of  our  programming  time  was 
spent  writing  new  programs. 

2.  Approximately  63%  of  our  time  was  spent  on 
modifying  old  programs. 

3.  Approximately  80%  or  1600  of  our  new  programs 
were  batch  sequential. 

4.  Approximately  65%  or  1300  of  our  new  programs 
were  required  in  less  than  three  weeks.  The 
"I  need  it  tomorrow"  syndrome. 
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A  large  percentage  of  our  programming  work  was 
redundant  in  nature  and  held  much  potential  for 
the  reusable  code  concept. 


PROCEDURE  USED  TO  TEST  OUR  REUSABLE  CODE  THEORY 


To  validate  the  conclusion  that  much  of  our  businass  appli¬ 
cations  programming  work  was  redundant  in  nature  and  held 
much  potential  for  the  reusable  code  concept ,  the  study 
team,  managers,  supervisors  and  key  personnel  discussed 
the  various  types  of  programs  written  in  our  organisation. 
They  came  up  with  the  following  types  and  terms: 

.  Edit  or  Select  -  This  consisted  of  editing 

Programs  data,  selecting  or  extracting 

data  and  reformatting  data. 
These  programs  are  classi¬ 
fied  as  utility  programs. 

.  Update1  Programs  -  This  consisted  of  updating 

or  modifying  data  on  any 
type  of  file. 

.  Report  Programs  -  This  Consisted  of  producing 

reports  from  any  type  of  file. 

Over  5000  production  COBOL  source  programs  were  classified 
by  type  using  the  following  procedure: 

Each  supervisor  was  given  a  list  of  the  programs  for  which 
he  was  responsible.  This  list  included  the  name  and  a 
brief  description  of  the  program  along  with  the  number  of 
lines  of  code.  The  supervisor  then  classified  each  program 
using  the  following  categories: 

.  Edit  or  selection  programs 

.  Update  programs 

.  Report  programs 

If  a  program  did  not  fall  into  the  above  three  categories 
then  the  supervisor  assigned  his  own  category  name.  The 
result  of  classification  analysis  by  program  type  was  as 
follows: 

1089  Edit  or  Selection  Programs 
1099  Update  Programs 
2433  Report  Programs 
247  Extract  Programs 
245  Bridge  or  Conversion  Programs 
341  Data  Fix  Programs 

5454  Total  Programs  Classified 


After  Analysis,  tha  24?  Extract  and  24 5  Bridga  or  Conver- 
sion  Programs  wars  placad  in  tha  Edit  or  Salaction  class 
and  tha  341  Data  Fix  Programs  wars  placad  in  tha  Update 
class.  Tha  results  of  our  program  classification  task 
aftar  these  Adjustments  wera  as  follows: 

1581  or  29%  of  our  programs  vara  Edit  or 
Salaction  Programs 

1440  or  26%  of  our  programs  wsre  Update  Programs 
2433  or  4 SI  of  our  programs  wars  Report  Programs 

5454  Total  Programs  Classified  aftar  Adjustments 

Tha  average  lines  of  coda  by  program  type  for  tha  5454 
programs  ware  as  follows: 

626  Linas  of  coda  par  Edit  or  Select  Programs 
788  Linas  of  coda  par  Update  Programs 
507  Linas  of  coda  par  Report  Programs 

Tha  supervisors  than  selected  over  50  programs  that  they 
fait  would  be  good  candidates  for  validating  our  redundancy 
theory.  Working  with  tha  supervisors,  tha  study  team  found 
that  approximately  40-60%  of  tha  COBOL  source  coda  in  tha 
programs  examined  was  redundant  and  could  be  standardised. 

In  September,  1976,  we  presented  tha  following  conclusions 
and  recommendations  to  management: 

CONCLUSIONS 

1.  Our  maintenance  and  productivity  problems  are  caused 
by  re-writing  the  same  redundant  functions  using 
different  programmer  styles  and  solutions. 

2.  Our  programming  environment  consists  of  three  types 
of  programs  and  that  40-60%  of  the  source  code  could 
be  standardized  and  pre-written  for  these  three  types. 

3.  The  reusable  module  source  code  technique  would  pro¬ 
vide  a  significant  increase  in  programmer  productivity 
and  provide  a  consistent  style  required  to  improve  our 
programming  maintenance  problem.  We  used  the  dock-to- 
stock  inventory  system  as  a  success  story. 

4.  Our  personnel  required  additional  training  to  upgrade 
their  skills.  We  used  the  skill  study  to  prove  our 
point . 


RECOMMENDATIONS 


1.  One  manager  and  five  kay  technicians  should  be 
assigned  to  staff  this  effort.  Each  location 
should  ba  rapresantad. 

2.  A  six  months  R&D  effort  should  ba  funded,  to 
develop  software  tools  and  aids,  standards, 
procedures  and  a  in-house  training  program. 

3.  A  one  year  implementation  period  should  ba 
scheduled  to  give  our  120  programmer  analysts 

S  days  of  training  and  follow-up  assistance  as 
required  in  reusable  coda  techniques. 

4.  A  steering  committee  should  be  established  to 
audit  the  progress  of  the  project. 

Managment  approved  our  proposal  with  one  provision.  They 

required  that  we  provide  for  measurement  rejorts  on  reusable 

code  results. 

OBJECTIVES 


The  primary  objective  of  the  Advanced  Software  Development 
(ASD)  project  was  to  reverse  the  upward  cost  trend  of  soft¬ 
ware  development  and  maintenance  by  using  reusable  modules 
and  logic  structures  and  implementing  new  disciplines  and 
aids  which  would  yield  a  more  reliable,  maintainable  product 
and  increase  productivity  of  our  systems  and  programming 
pt . sonnel. 

Other  goals  which  complemented  the  above  objective  were  to: 

o  Design,  create  and  reuse  standard  modules  for 
new  applications  development.  60%  reusability 
was  the  target. 

o  Increase  the  maintainability  of  all  new  programs 
and  programs  that  require  modifications. 

o  Increase  the  control  over  our  work  products. 

o  Decrease  the  overhead  required  to  maintain  our 
work  products. 

o  Upgrade  the  skills  of  our  staff  using  state  of 
the  art  technology. 

o  Create  an  awareness  and  environment  that  fosters 
the  development  of  new  tools  and  aids  within  the 
organization. 
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IMPLEMENTATION. OF  THIS  CONCEPT 


General 

Encouraged  by  the  results  of  our  study  we  decided  to 
further  test  the  concept  by  proceeding  in  parallel  along 
two  paths.  One  path  was  the  use  of  pre-written  COBOL  copy 
code  source  modules.  The  other  path  was  to  develop  pre¬ 
written  COBOL  logic  structures  that  contain  redundant 
programming  functions. 

Pre-Written  Modules 

TheSe  are  modules  which  are  written  for  a  specific  purpose. 
They  are  walked  through,  tested,  certified,  documented  and 
stored  in  a  library.  Examples  in  a  scientific  environment 
would  be  routines  for  trigonometric  functions.  In  a 
business  applications  environment  we  break  these  into  three 
groups.  The  first  group  is  called  Universal  Routines. 

These  are  routines  that  most  companies  can  use.  Examples 
are : 

-  Gregorian  date  edit  routine. 

-  Gregorian  to  Julian  date  conversion  routine,  and 
vice  versa. 

-  Payroll  tax  routines. 

The  second  group  is  called  Company  Routines.  These  are 
routines  that  are  used  for  our  standard  systems.  Examples 
are : 

Part  number  validation  routine. 

-  Manufacturing  day  conversion  routine. 

Edits  for  data  fields  used  throughout  our  standard 
company  systems,  e.g.  employee  number,  purchase 
order  number,  etc.. 

The  third  group  is  called  Functional  Area  Routines.  These 
are  routines  that  are  used  by  functional  areas  in  our 
company  such  as  manufacturing  or  accounting.  Examples  are: 

-  Customer,  vendor,  and  invoice  number  routines. 


For  those  companies  doing  work  on  Government 
Contracts  there  would  be  another  group.  Many 
data  fields  are  standardized  for  all  Government 
applications.  Examples  are  Federal  Stock  Number, 

Federal  Vendor  Code,  Interchangeability  Code, 

Shelf  Life,  etc. 

From  a  systems  and  programming  point  of  view,  all  these 
routines  fall  into  several  categories.  What  follows  are 
the  descriptions  of  the  categories  and  the  number  of  modules 
that  exist  to  date  in  our  reusable  code  library. 

-  File  description  i.e.  FDs 

-  Record  descriptions  i.e.  01  levels  in  an  FD 
or  in  working  storage 

-  Edit  Routines  i.e.  the  data  area  and  procedure 
code  to  edit  a  specific  data  field 

-  Functional  Routines  i.e.  the  data  area  and 
procedure  code  to  perform  some  function,  such 
as  left  justify  and  zero  fill 

-  Data  Base  I/O  area  specifically  IBM's  IMS  I/O 
area  descriptions 

ii  ' 

Data  Base  interface  module,  specifically  IMS 
Program  control  blocks 

-  Data  Base  Search  Arguments,  specifically  IMS 
Segment  Search  Arguments 

Data  Base  Procedure  Division  Calls 

Using  the  reusable  code  centralized. library  approach,  eight 
major  systems  have  been  developed  averaging  over  60%  reusable 
code  during  the  last  four  years. 

KEY  CONCEPT  BEHIND  OUR  REUSABLE  CODE  SUCCESS 

The  key  concept  behind  this  successful  methodology  is  that 
reusable  modules  ai*e  designed,  coded,  tested,  certified 
and  documented  prior  to  completing  programming  specifications. 
When  a  programmer  receives  a  program  specification  it  spec¬ 
ifies  what,  why,  when  and  how  to  use  modules  that  are  applicable 
to  the  program  problem.  The  technique  resembles  a  manufacturing 
environment  in  that  many  standard  parts  are  used  in  building 
manufactured  products.  The  only  difference  in  our  environment 
is  the  program  is  the  product.  This  approach  produces  systems 
that  are  mors  reliable  with  less  testing,  coding  and 


documentation.  The  biggest  payback  however,  lies  in 
greatly  reduced  maintenance  since  a  reusable  module  is 
independent  of  the  physical  program  and  need  only  be 
modified  once  even  though  it’s  used  in  many  programs. 

LOGIC  STRUCTURES 

A  logic  structure  has  a  pre-written  Identification  Division, 
Environment  Division,  Data  Division,  and  Procedure  Division. 

It  is  not  a  complete  program  because  some  paragraphs  contain 
no  code,  and  some  record  descriptions  are  also  empty,  con¬ 
sisting  only  of  the  01  level.  It  does,  however,  contain, 
many  complete  01  levels  and  procedure  paragraphs. 

We  have  written  six  Logic  Structures: 

The  update  is  designed  for  the  classical,  sequential  update. 
There  is  a  version  with  an  embedded  sort  and  a  version 
without.  The  update  is  designed  for  situations  where  the 
transaction  record  contains  a  transaction  type  field  (add, 
change  or  delete).  This  can  be  easily  overridden  for  co¬ 
llator  type  updates.  The  update  logic  structure  is  also 
designed  to  accommodate  multiple  transactions  per  master 
record.  Error  messages  to  a  transaction  register  are 
provided  for  standard  errors  such  as  an  attempt  to  add  an 
already  existing  record.  Final  totals  are  also  provided, 
as  well  as  sequence  checking. 

The  report  logic  structure  is  also  written  in  two  versions, 
one  with  and  one  without  a  sort  of  the  input  records  prior 
to  report  preparation.  Major,  intermediate  and  minor  levels 
of  totals  are  provided  for,  but  more  may  be  added  if  needed. 
If  multiple  sequences  of  reports  are  desired  the  record  can 
be  released  to  the  sort  with  multiple  control  prefixes. 
Paragraphs  are  also  provided  for  editing,  reformatting,  and 
sequence  checking. 

The  select  logic  structure  is  also  written  in  two  versions, 
with  or  without  a  sort  of  the  input  records.  This  logic 
structure  was  designed  for  two  purposes.  One  is  the  editing 
of  input  records.  In  effect,  the  input  records  are  examined 
based  on  some  criteria  and  written  to  the  selected  (good 
records)  or  non-selected  (error)  file.  Another  use  for  this 
logic  structure  is  the  selection  of  records  from  a  file, 
based  on  some  criteria,  for  later  use  in  a  report.  In 
practice  one  of  its  most  common  uses  has  proven  to  be  special 
purpose  runs  against  master  files  when  customers  request 
specific  non-standard  modifications  to  existing  data. 
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Combinations;  We  rank  these  logic  structures  in  the 
order-update-report- select-  when  determining  which  one 
should  be  used  for  a  combined  program.  The  highest 
included  function  determines  which  logic  structure  to 
use.  For  instance,  an  update  with  editing  of  transactions 
would  indicate  adding  edit  code  to  the  update,  rather  than 
adding  update  logic  to  a  select. 

CONSTRUCTION  OF  LOGIC  STRUCTURES 


For  each  type  of  Logic  Structure  there  is  a  central 
supporting  paragraph. 

For  the  update  program  it  is  the  high-low-equal 
comparison . 

For  the  report  program  it  is  the  paragraph  that 
determines  which  level  of  control  break  to  take. 

For  the  selection  program  it  is  the  select/non¬ 
select  paragraph. 

Let  us  consider  the  report  program  as  an  example. 

Prior  to  the  control  break  paragraph  we  can  identify  support 
functions  which  must  occur  in  order  for  the  control  break 
paragraph  to  function.  Examples  are:  get-record,  sequence- 
check-record,  edit-record-prior-to-sort ,  and  build-control- 
keys.  These  are  supporting  functions.  Other  functions  such 
as  major-break,  intermediate-break,  minor-break,  roll- 
counters,  build-detail-line,  print -detail-line ,  page-headers, 
etc.  are  dependent  on  the  control  break  or  central  paragraph. 

Obviously  many  of  these  paragraphs  (functions)  can  be  either 
completely  or  partially  pre-written. 

Our  report  program  Logic  Structure  Procedure  Division 
contains  15  paragraphs  in  the  version  without  a  sort,  and 
20  paragraphs  in  the  version  with  a  sort. 

To  further  clarify  what  we  mean  when  we  talk  about  logic 
structures,  it  might  be  helpful  to  indicate  the  number 
of  non-comment  lines  of  code  in  each  part  of  a  Report 
Logic  Structure,  without  an  embedded  sort. 
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Identification  Division 
Environment  Division 


Data  Division 


File  Section 


Working  Storage  Section 


01 

AA1  - 

01 

BB1  - 

01 

BB2  - 

01 

BB4  - 

01 

CC1  - 

01 

DD1  - 

01 

EE1  - 

01 

FF1  - 

01 

GG1  - 

01 

GG2  - 

01 

GG3  - 

01 

HH1  - 

01 

LL1  - 

01 

SSI  - 

01 

TT1  - 

01 

TT2  - 

01 

TT3  - 

01 

TT4  - 

CARRIAGE -CONTROL-SPACING 
CONSTANTS -AREA 
TRANSACTIONS -STATUS 
FILES -STATUS 
COUNT -AREA 
MESSAGE -AREA 
TRANSACTION-READ -AREA 

KEY -AREA 
HEAD-LINE1 
HEAD-LINE2 
HEAD-LINE 3 
DETAIL-LINE 
TOTALS -AREA 
SUBSCRIPT -AREA 
TOTAL-LINE -MINOR 
TOTAL-LINE-INTER 
TOTAL- LINE -MAJOR 
TOTAL-LINE -FINAL 


Procedure  Division 


0010  -  INITIALIZE 
0020  -  MAIN-FLOW 
0030  -  WRAP-IT-UP 
0040  -  CHECK-CONTROLS 
0050  -  FINAL-BREAK 


0060  -  MAJOR-BREAK 

0070  -  INTER-BREAK 

0080  -  MINOR-BREAK 

0090  -  PRINT-TOTAL 

0100  -  ROLL-COUNTERS 

0110  -  FILL-DETAIL-LINE 

0120  -  WRITE -PRINT -LINE 

0130  -  NEW-PAGE -HEADING 

0140  -  GET-TRANSACTION -RECORD 

0150  -  TRANSACTION-FORMAT-OR-EDIT 

0160  -  SEQUENCE -CHECK 
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Only  two  of  these  paragraphs  are  totally  empty.  Some, 
like  G6 2 -HEAD- LINE 2  have  a  filter  to  prevent  Compute 
errors  if  they  are  not  needed,  but  they  are  essentially 
empty.  Taken  together  as  a  program  they  provide  the 
programmer  with  a  modular  functional  structure  on  which 
he  can  build  a  report  program,  many  parts  of  which  are 
partially  pre-written. 

For  the  update  logic  structure  the  central  paragraph  is 
the  hi-low-equal  control  paragraph.  Prior  to  the  central 
control  paragraph  there  must  be  supporting  functions  such 
as  get-transaction,  sequence-check-transaction,  edit- 
transaction,  sort-transaction,  get-master,  sequence-check- 
master,  build-keys,  etc.  As  a  result  of  this  central 
paragraph  you  will  have  functions  such  as  add-a-record , 
delete-a-record,  change-a -record ,  print -activity -register , 
print-page-heading,  print-control-totals,  etc. 

Our  update  logic  structure  Procedure  Division  contains  22 
paragraphs  in  the  non-sort  version  and  26  paragraphs  in 
the  version  with  an  embedded  sort. 

BENEFITS  OF  REUSABLE  CODE 

We  believe  that  logic  structures  have  many  benefits. 

-  They  help  clarify  the  programmer's  thinking  in 
terms  of  what  he  is  trying  to  accomplish. 

-  They  make  walk-thru  easier. 

-  They  help  the  analyst  communicate  with  the 
programmer  relative  to  the  requirements  of 
the  system. 

-  They  facilitate  debugging. 

-  They  eliminate  certain  error  prone  areas  such 
as  end  of  file  conditions,  since  the  logic  is 
already  built  and  tested. 

-  They  reduce  program  preparation  time  since  parts 
of  the  design  and  coding  are  already  done. 

However,  we  believe  the  biggest  benefit  comes  after  the 
program  is  written,  when  the  user  requests  modifications 
or  enhancements  to  the  program.  Once  the  learning  curve 
is  overcome,  and  the  programmers  are  familiar  with  the  logi 
structure,  the  effect  is  similar  to  having  team  programming 
with  everyone  on  the  same  team.  When  a  programmer  works 
with  a  program  created  by  someone  else  he  finds  very  little 
that  appears  strange.  He  does  not  have  to  become  familiar 
with  another  person's  style,  because  it  is  essentially 
his  3tyle. 


STRUCTURED  TECHNOLOGY  GUIDELINE 

Our  structured  design  approach  advocates  that  a  program 
be  divided  into  segments  called  functional  modules.  Each 
of  these  modules  is  independent  of  other  modules  in  the 
program  and  performs  a  separate  function.  Emphasis  is 
placed  on  the  main  functional  modules  first  because  they 
contain  the  highest-level  control  logic  and  are  the  most 
critical  to  the  success  of  the  program.  These  modules 
are  designed,  coded,  and  tested.  Then  the  next  lower- 
level  modules  are  created  in  the  same  manner.  In  this 
way,  most  of  the  details  of  the  design  are  left  until 
the  lowest-level  modules  are  designed.  Thus,  a  program, 
develcped  by  using  a  top-down  design  consists  of  a 
series  of  modules  created  and  related  in  a  treelike 
(hierarchical)  structure. 

The  treelike  structural  relationship  between  modules  in 
a  top-down  designed  program  can  be  represented  in  a 
STRUCTURE  CHART .  The  structure  chart  shows  each  module 
and  its  functional  relationship  to  other  modules.  The 
flow  of  control  is  from  the  highest-level  module  to  the 
lowest-level  modules  (top-down).  Each  module  is  called 
or  invoked,  by  a  next-higher-level  module. 

The  logic  structures  described  earlier  in  the  report 
reflect  this  kind  of  design. 

STRUCTURED  CODING  GUIDELINE 

Pure  structured  program  code  reflects  three  basic  control 
structures : 

Sequence  (ADD  A  TO  B,  MOVE  C  TO  D,  ETC) 

Selection  (IF  A  less  than  B  MOVE  C  TO  D  else 
MOVE  D  TO  E) 

Looping  (PERFORM  ROUTINE  A  UNTIL  THERE  IS 
NO  MORE  DATA) 

Each  of  these  structures  has  a  single  entry  and  single  exit. 
The  program  listing  is  easy  to  read  because  there  is  no 
random  branching  using  GO  TO's  (FORWARD  OR  BACKWARD)  from 
one  part  of  the  program  to  another  part.  Since  control  flows 
from  top  to  bottom,  the  listing  can  be  read  more  or  less  like 
a  book . 


Our  approach  advocates  a  modified  approach  to  the  pure 
structured  coding  defined  above.  The  approach  advooated 
using  00  TO's  that  branch  to  the  entrv,  or  exit  points 
of  a  functional  module. 

STRUCTURED  WALKTHROUGH  GUIDELINE 

The  purpose  of  STRUCTURED  WALKTHROUGHS  is  to  find  errors 
and  to  find  tham  at  the  earliest  possible  time  by  having 
one  or  more  peer  people  review  the  software  work  products 
of  others. 

The  advantages  of  walkthroughs  are  many:  higher  reliability, 
increased  maintainability,  better  dissemination  of  techni¬ 
cal  information  between  technicians,  and  an  increased 
likelihood  that  work  can  be  salvaged  if  someone  leaves 
the  project. 

The  walkthrough  technique  used  in  our  organization  is 
informal.  Usually  it  consists  of  one  person  and  the  author. 
The  objective  is  to  find  errors,  not  to  criticize  style. 

STANDARD  COBOL  PROGRAM  FORMATS 

The  purpose  of  STANDARD  COBOL  FORMATS  is  to  improve  the 
readability  and  understandability  of  our  primary  work 
product  -  PROGRAMS. 

Our  organization  modifies  over  2,000  programs  annually. 

In  addition  to  this,  over  1,200  new  programs  are  written 
by  practicing  programmers  in  our  organization.  Most  of 
this  activity  centers  around  enhancing  existing  systems 
♦hat  have  serviced  the  user  community  for  many  years. 

ost  of  the  programmers  who  now  maintain  these  old  systems 
are  not  the  original  authors.  Since  programmers  must  read 
nd  understand  programs  before  they  can  fix  or  enhance  them, 
a  process  was  developed  to  improve  maintainability. 

To  satisfy  this  objective  the  projedt  team  modified  soft¬ 
ware  that  produces  a  consistent  standard  COBOL  formatting 
style.  Some  of  the  primary  functions  of  the  software  are: 

o  Attaches  paragraph  numbers  to  paragraph  names. 

This  increases  the  productivity  of  maintenance 
programmers  because  logic  flow  is  easier  to 
follow. 

o  Provides  indentation  for  the  "If  Then  Else" 
construct  in  COBOL. 


o 


Prints  one  COBOL  statement  per  line. 


o  Provides  spacing  between  paragraph  names. 

o  Assigns  consistent  hierarchical  numbers  to 
group  and  subordinate  data  names  in  the  data 
division. 

MECHANICS 

The  procedure  to  generate  a  STANDARD  COBOL  FORMAT  is  as 
follows : 

o  Before  the  program  is  moved  back  to  the 

CENTRALIZED  PRODUCTION  SOURCE  LIBRARY,  the 
programmer  runs  the  program  through  the 
STANDARD  COBOL  FORMAT  SOFTWARE  using  a 
standard  procedure. 

The  implementation  of  this  technique  addressed  the  problem 
of  what  to  do  with  7,000  programs  written  over  a  period 
of  10  -  15  years  by  many  programmers  using  different  writing 
styles. 

CENTRALIZED  LIBRARY  CONCEPT 

The  centralized  source  program  library  at  the  Business 
Computer  Center  at  West  Andover,  Massachusetts  is  a 
collection  of  functional  libraries  that  are  shared  in  common 
among  several  independent  customers.  The  libraries  are  used 
to  store  and  modify  both  test  and  production  source  programs 
and  copy  code.  Each  library  in  the  network  is  functional 
in  the  respect  that  it  exists  to  satisfy  a  specific  need. 

BENEFITS  OF  A  COMMON  LIBRARY  SYSTEM 

A  centralized  facility  offers  its  customers  and  the  Division 
the  following  advantages  over  a  diversified  multiple  library 
system . 

COST  REDUCTION 

A  single  common  facility  is  maintained  by  full  time  indivi¬ 
duals  who  have  expertise  in  library  management.  This  is 
preferable  to  the  situation  where  individuals  are  required 
ro  expend  man  hours  on  an  on-going  basis  to  maintain  their 
j«n  private  libraries  and  whose  primary  responsibility  is 
'jomething  other  than  library  overhead  maintenance  tasks. 

A  second  significant  cost  reduction  is  the  elimination  of 
redundant  and  overlapping  facilities. 


INCREASED  PRODUCTIVITY 


Because  application  programmers  do  not  have  to  spend  time 
maintaining  private  libraries,  they  have  more  time  to 
perform  additional  programming  oriented  functions.  This 
tends  to  relieve  the  application  programmer  from  perform¬ 
ing  clerical  activities  and  many  peripheral  programming 
duties  are  reduced  to  standardized  tasks. 

SECURITY  ENHANCED 


Library  management  by  full  time  experienced  professionals 
greatly  reduces  the  likelihood  of  modules  or  libraries 
being  permanently  damaged  or  lost  due  to  human  error 
and/or  inadequate  backup  facilities. 

GLOBAL  PROCESSING 


Because  all  source  code  is  systematically  stored  in  one 
centralized  location,  it  is  possible  to  perform  a  variety 
of  global  functions. 

EXAMPLES 

1<  Scanning  all  lines  of  all  modules  on  the 
library  for  specific  character  string. 

2.  Making  global  changes  to  all  (or  selected 
modules)  on  the  library. 

3.  Insuring  that  all  organizations  within  the 
division  are  complying  with  established 
standards . 

OUR  MEASUREMENT  APPROACH 


Since  the  introduction  of  the  Structured  Technologies,  the 
data  processing  community  has  heard  claims  of  increases  in 
programmer  productivity.  These  claims  range  from  moderate 
to  fantastic  and  tend  to  intensify  the  concern  many  install 
ations  have  about  their  own  productivity.  "Will  the 
Structured  Technology  significantly  improve  my  programmers' 
productivity  and,  if  so,  by  how  much?",  is  a  question  many 
data  processing  managers  are  asking  themselves. 

Unfortunately,  most  installations  have  no  real  baseline 
from  which  their  own  productivity  increases  can  be  measured 
In  fact,  very  few  know  what  to  measure,  much  less  how  to 
measure  it.  At  the  present  time,  there  are  no  industry 
standards  for  measuring  either  the  pertinent  aspects  of 
software  development  or  the  people's  time  that  goes  into 
producing  software.  Therefore,  any  attempt  to  compare 
one  installation’s  performance  against  another,  may  be 
futile  if  not  downright  dangerous. 
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Perhaps  the  greatest  danger  of  all  is  getting  caught 
up  in  the  magic  of  measurement  for  its  own  sake.  Some 
installations  have  started  to  count  lines  of  code  and 
keep  track  of  programming  time  without  really  determining 
why  this  data  was  needed  or  how  it  would  be  used.  In 
effect,  they  are  making  measurement  an  end  rather  than 
a  tool  to  be  used  to  help  meet  their  goals. 

With  these  considerations  in  mind ,  it  was  the  purpose  of 
our  software  productivity  to  project  to  identify  factors 
that  should  be  considered  in  a  measurement  effort. 

Our  Measurement  Approach  in  no  way  tried  to  determine 
THE  SINGLE  WAY  TO  MEASURE  for  no  such  simplistic  solu¬ 
tion  currently  exists. 

MEASUREMENT  PHILOSOPHY 


At  the  start  of  the  project,  it  was  decided  by  manage¬ 
ment  and  our  project  team  that  it  would  be  counter¬ 
productive  to  our  productivity  goals  to  have  programmers 
keep  manual  records  or  rely  on  their  memories  for  reusable 
code  statistics.  Instead,  we  decided  to  measure  reusable 
code  utilization  at  the  program  and  programmer  level. 

These  statistics  are  produced  when  a  program  is  moved 
from  our  test  source  library  to  our  production  source 
library.  This  approach  made  the  gathering  of  statistics 
transparent  to  our  managers,  supervisors  and  programmers. 
In  addition,  it  gives  management  and  supervisors  the 
visibility  to  monitor  their  reusable  code  progress. 

MEASUREMENT  PLAN  AND  OBJECTIVE 


Our  objective  was  to  produce  reliable,  maintainable 
programs  and  increase  the  productivity  of  our  practicing 
programmers  by  providing  40  -  60%  reusable  code  for  new 
programs.  It  was  felt  that  this  was  attainable  because 
of  our  success  with  the  reusable  code  concept  in  test 
mode  in  1977.  During  this  time,  a  range  of  15-85%  reusable 
code  was  realized  for  new  programs  when  logic  structures 
and  copy  code  were  used. 

Our  plan  for  implementing  our  reusable  code  technique 
was  composed  of  three  simple  steps. 

1.  Establish  a  flexible  standard  which  made  it 
mandatory  that  programmers  would  use  logic 
structures  and  copy  code  unless  they  could 
tell  us  why  they  should  not.  This  approach 
allowed  us  to  gain  the  feedback  from  our 
programmers  necessary  to  improve  our  reusable 
code  technique. 
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2. 


Give  our  programming  personnel  five  days 
training.  This  was  accomplished  by  using 
half  day  training  sessions  and  by  gibing 
assistance  to  each  programmer  as  required 
after  the  training  sessions  were  completed. 

3.  By  establishing  a  measurement  system  to 
measure  the  utilization  of  reusable  code 
by  program  and  programmer. 

The  major  items  measured  were  by  program*  programmer  super 
visor  and  plant.  Other  items  reported  on  were: 

1.  Total  lines  of  code  for  each  new  program. 

2.  Total  reusable  copy  code  for  each  new  pro¬ 
gram  in  terms  of  modules,  lines  of  code  and 
reusable  code  percentage. 

3.  Total  number  of  logic  structures  used  and  total 
number  of  reusable  lines  of  code  within  each 
logic  structure,  i.e.,  code  that  the  programmer 
did  not  have  to  modify. 

REUSABLE  CODE  RESULTS  TO-DATE 


By  eliminating  redundant  programming  and  application  depen 
dent  functions,  the  following  results  have  been  attained 
over  a  period  of  four  years: 

o  8  New  Systems  have  averaged  over  60% 

Reusable  Code 

o  Over  3000  New  Programs  averaged  over  40% 

Reusable  Code 


o  Over  6000  Programs  are  now  more  maintainable 

o  Over  2500  Reusable  Modules  now  exist  in  a 
Central  Library 

HOW  WE  APPRAISE  OUR  PROGRESS 


Measurement  reports  are  distributed  weekly  to  all  super¬ 
visory  and  management?  personnel  at  each  location.  This 
gives  our  managers  and  supervisors  the  opportunity  to 
analyse  and  appraise  their  progress  and  the  progress  of 
others. 


To  control  our  progress  end  deal  with  problems  of  imple- 
mentation,  quarterly  status  meetings  are  held  with  project 
managers  and  management.  These  meetings  give  us  meaningful 
feedback  required  to  tune  our  reusable  code  technique.  In 
addition,  these  meetings  also  give  recognition  to  our 
programming  personnel  for  their  reusable  code  feedback, 
utilization  and  contributions. 

SUPPORT  PRODUCTIVITY  TOOLS.  AIDS  AND  SERVICES 

Other  disciplines  and  aids  developed  and  implemented  by 
the  ASD  project  team  used  to  complement  REUSABLE  PROGRAM 
MODULES  -  STRUCTURED  PROGRAMMING  TECHNOLOGIES  -  STANDARD¬ 
IZED  COBOL  FORMATS  -  CENTRALIZED  PROGRAM  LIBRARIES  -  are: 

o  Indexing  Scheme  to  foster  the  usage. of  reusable 
code  i 

o  Programming  Standards  to  provide  consistency 
required  to  support  the  reusable  code  concept. 

o  Where  Used  System  to  monitor  and  analyze 
reusable  code. 

o  Certification  Procedures  to  validate  reusable 
code. 

o  Program  Change  Activity  system  to  provide 
change  control  visibility. 

o  Configuration  Control  System  to  provide 

synchronization  of  source  and  load  programs. 

o  Standard  Job  Control  Language  Procedures  to 
improve  productivity.  - 

o  Entry  Level  Training  Program  -  Entry  level 
programmers  are  put  through  a  two  month  full 
time  reusable  code  training  program.  They 
design  a  complete  inventory  system  using 
structured  technology  and  reusable  code  techni¬ 
ques.  In  addition,  they  receive  training  in 
productivity  tools  and  aids  and  reach  a  high 
level  of  productivity  in  three  months  after  the 
training  is  completed.  The  students  use  a  team 
approach  with  the  instructor  acting  as  the  chief 
technican. 

CONCLUSION 


After  studying  our  business  programming  community  for  over 
four  years,  we  have  concluded  that  we  do  similar  work  year 
in  and  year  out  and  much  of  this  work  deals  with  redundant 
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programming  functions.  By  standardizing  those  functions 
in  the  form  of  reusable  code  and  logic  structures ,  gains 
in  productivity  and  reliability  can  be  attained  and  the 
programmers  can  concentrate  on  the  real  creative  problem 
rather  than  the  redundant  one.  In  addition  to  the  one 
time  development  benefit,  the  programming  community  can 
eliminate  much  of  the  programmer's  individual  styles 
which  are  the  root  of  the  maintenance  dilemma  that  most 
business  data  processing  installations  are  faced  with 
today . 

FUTURE  PLANS 

Currently,  our  logic  structure  programs  provide  us  on  the 
average  40-60%  reusable  code  for  new  programs.  The 
programmer  however,  must  add,  change  and  delete  code  to 
produce  a  complete  program  to  fulfill  the  problem 
specifications. 

Examples  are: 

-  Moving  data  fields  from  the  input  area  to 
the  detail  line  image  in  the  report  logic 
structure . 

-  Moving  data  fields  into  the  hi  -  lo  -  equal 
key  in  an  update  logic  structure. 

To  attain  additional  productivity  the  next  logical  extension 
is  to  automate  these  changes  rather  than  have  the  programmer 
manually  make  the  modifications. 

We  are  currently  developing  software  which  will  automate 
our  logic  structure  technique  by  using  an  INTERACTIVE  COBOL 
Generator  process  which  will  automatically  generate  the 
COBOL  source  code  necessary  to  complete  a  large  percentage 
of  our  new  programs  utilizing  structure  code. 

For  example,  the  question  "How  many  levels  of  control  breaks?" 
would  cause  the  required  number  of  control  break  paragraphs 
to  be  included  in  the  source  code.  In  response  to  a  subse¬ 
quent  question  such  as  "What  data  field  controls  the  most 
major  break?"  could  cause  that  data  name  to  replace  the  word 
"major"  in  all  references  to  that  paragraph.  Thus,  the 
paragraph  "0090-Major-Break"  becomes  "0090-Department-Break". 

A  sentence  which  reads  "Perform  0100-Inter-Break"  would  read 
"Perform  0100-Emp ioyee-number-break" . 
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This  interactive  prompting  approach  will  enable  a  programmer,, 
an  analyst  or,  in  some  cases,  even  a  user  to  produce  a  work¬ 
ing  COBOL  source  program  in  less  than  30  minutes  in  one 
interactive  session.  This,  coupled  with  our  powerful  text 
editing  capability  and  other  tools  and  aids  will  provide  our 
programmers  with  a  full  set  of  tools  to  minimize  the  amount 
of  time  required  to  bring  an  application  into  production. 

/; 


i 
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